Evaluation Results

Evaluation result – Adult Education Academy 2024

Project-coordinator: Prof. Dr. Regina Egetenmeyer (Julius-Maximilian University Würzburg)

Evaluation: Professorship for Adult and Continuing Education, Adult Education Academy Team (Frank Schenk, Julia Saam)

The most important results of the evaluation of the Adult Education Academy 2024 are summarised below. The 11th Adult Education Academy - AEA for short - took place in 2024. The AEA has been evaluated using the same set of questions since 2016, which allows for a good comparison between years.

The evaluation includes an analysis of all participants, whereby a breakdown is sometimes made between the following three different subgroups: Master's students, doctoral students, and practitioners in the field of adult education and lifelong learning. The questionnaire combines dichotomous yes-no questions and rating scales from 1 to 5 (1 being the lowest and 5 the highest). In addition, open questions were used. In the following, the results of the group sample, the overall assessment as well as the effects and outcomes will be presented.

Participants´ profile

At first, the evaluation looks upon some characteristics of the sample. The following information will provide an overview of some of the most important data of the group:

• That year 55% (44 people) of the participants are Master’s students. A further 22.5% (18 people) are enrolled as doctoral students at the time of the Academy. 12 (15%) of the participants are practitioners form the field of adult education. The last 7.5% (6 persons) are either staff at a university, moderators, or enrolled in another programme to participate in the AEA.
• The international nature of the Academy programme is repeatedly demonstrated by the wide repertoire of different countries in which students have graduated or will graduate. 8 of the 20 countries mentioned are European countries. The remaining countries are located on many different continents. The most represented country from another continent was India, from which 8 participants came that year.
• Of the 79 responses regarding their own gender, 62% (49 people) stated that they are female. The remaining 30 (38%) are male participants, while no one identified as divers.
• For the following question, participants are asked on a scale of 1 to 5 what motivated them most to attend the Academy. The top answer is learning from lecturers from different countries (mean 4.61), while meeting students from other countries is only just behind (mean 4.6). Improving one's own career prospects is the next following answer (mean 4.58). The ranking after these answers are "comparative studies" (mean 4.47) and "international and European strategies for lifelong learning" (mean 4.34). Meeting practitioners is of slightly less importance to the participants (mean 4.31). Improving English language skills (mean 3.78) and travelling to Germany (mean 3.74) has the lowest relevance for the participants.
• Most participants (67.5%) become aware of the Academy because of the recommendation by their professors. This especially applies for the master students (81.8%). The search for an international study mobility is another relevant category for the attendance of the Academy (44.6% of the participants). The lowest ranked reason for attending the Academy this year is that somebody attended the programme before (10.8%).
• On a scale from 1 to 5, the most kind of support from the home university is motivational support (e.g. encouraging to participate, supporting, self-confidence etc.) to attend the Academy (mean 4.42), while financial support (mean 3.05) is given less.
• For more than half of the students (57.8%) the Academy 2024 is the first study mobility. The most participants, that have already completed a study mobility, have spent a semester abroad (19.8%), which is particularly evident among Master's students (27.3% of all master students).
• The mean of all participants, about the question on how intensively they have dealt with the topics of the AEA before, is at 3.51. It must be highlighted that the Master's students have a mean of 3.07, while mean of the practitioners is much higher with 4.09. The doctoral students are between those two groups, but still above the total mean, with 3.88.
• The previous interest in the topics on the other hand is much higher, with 80.7% of all participants showing interest in adult education in general. The interest in trans- and international comparative adult education is noticeably lower, but still over half (57.8%) of the participants stated that they are interest in this topic.
• The level of competence in some categories before the Academy is also analysed. Here, the top competence of the participants seems to be English language (72.3%), while the interaction skills with people from other countries are about 10% lower (61.5%). This shows that the participants have confidence in their way of communicating with other students and lecturers. In terms of analytical and methodological skills for conducting research, they rate their own skills lower (analytical 56.6%, methodological 42.2%). The lowest value is found for the understanding of ALE in other countries (32.5%).
• The last category concerning the sample group asks about the career plans and aspirations. This of course does not concern the practitioners. 28% of the master students indicated their interest in the practice, but also an academic/research career in ALE. 25% of the Master’s students on the other hand only show interest in a career in the practice of ALE. While 23% did not yet decide, which career path they would like to go, other 22% are looking towards taking up PHD/doctoral studies. Only one of the Master’s students aims an academic/research career in ALE. Looking at the doctoral students, 57% focus a career in practice of ALE. Other 31% aim an academic/research career and a career in practice, while the remaining 2 persons would like something else, or have not yet decided.

Assessment of the programme

The overall evaluation of the programme was rated by the participants on a 5-point scale (figure 1). The mean of the practitioners is the highest with 4.8. The mean in total at 4.56 is slightly above the mean of the Master’s students with 4.43. The doctoral students are in the middle between the Master’s and practitioners with a mean of 4.65.

Fig. 1: Overall rating of the Adult Education Academy 2024

As a next part of the evaluation of the whole programme, the different modules of the Academy are evaluated upon their organizational, academic and didactical quality (figure 2). The best score got the Freirean studies in the 1^st week (mean 4.6), while the preparatory phase got the lowest score (mean 4,24).

Fig. 2: Organisational, academic, didactical quality of the AEA 24 programme modules

Effects and outcomes of the Adult Education Academy

Effects on the interest in the topics are presented in figure 3. While the effect on the motivation to focus on trans-/international aspects in further studies is the lowest with a mean of 4.3, the interest in adult education and lifelong learning in general shows the greatest effect with a mean of 4.57.

Fig. 3: Effects of the AEA 24 on the participants´ thematic interest

When asked about the motivation to undertake further study mobilities after the Academy, all but one participant chose either a 4 or a 5, which lead to a mean of 4.63. While the interest of the Master’s students for taking up a PHD is relatively low with a mean of 3.56, the doctoral students show much more interest in postdoc research with a mean of 4.71.

Following now in figure 4, the improvement of the participants skills is shown. The improved understanding of ALE in other countries shows the highest results with a mean 4.66, which is closely followed by the change of perspective of ALE in the own country, which has a mean of 4.61. The lowest improvement can be seen in the English language competencies, which has a mean of 4.04.

Fig. 4: Effects of the AEA 24 on the participants´ competences

At last, the personal, academic and (only for practitioners) employment outcomes of the participants will be presented. Those categories will be differentiated between Master’s students, doctoral students, and practitioners.

• The personal (mean 4.37) and academic (mean 4.32) outcomes of the Master’s students show great similarities. As only one person of the Master’s students has given a differing answer which thereby lead to the difference between the two outcomes, it can be overall said that the Master’s students considered the Academy to be equally influential on both academic and personal improvement.

• The same can be seen for the doctoral students, where the personal (mean 4.53) and the academic (mean 4.59) outcome have similar means.
• Looking at the practitioners a slightly greater difference can be recognised between the academic (mean 4.56) and the personal (mean 4.78) outcomes. However, as the group of practitioners was with 9 people quite small, the bigger difference can again be traced back to one person choosing a lower ranking in academic outcomes than the others. Looking at the outcome for employment (mean 4.6), this result is close to the results of the other two categories.

Hybrid setting

This year the Academy was executed in a hybrid setting. This means that both weeks were mostly held in a present setting, field visits were made in Würzburg, as well as outside of Würzburg (Frankfurt and Nürnberg). However, there was the opportunity to also attend online, as two comparative groups and one group during the first week, including online field visits, were designed for online and on campus participants. The hybrid setting gave people, which were not able to travel to Würzburg because of several different reasons, the opportunity to still attempt the programme fully.

Evaluation result – Adult Education Academy 2023

Project-coordinator: Prof. Dr. Regina Egetenmeyer (Julius-Maximilian University Würzburg)

Evaluation: Professorship for Adult and Continuing Education, Adult Education Academy Team (Frank Schenk, Julia Saam)

A summary of the most important findings of the evaluation of the Adult Education Academy 2023 will be presented in the following. The 10^th Adult Education Academy -short AEA- was held in 2023. Since 2016 the AEA is evaluated based on the same set of questions, which should lead to a good comparison between the years.

The evaluation will include a view on all participants, while sometimes a division is made, between the three different subgroups: master students, doctoral students and practitioners in the field of adult education and lifelong learning. The topics of the group sample, overall assessment, the effects and the outcomes will be looked upon.

The questionnaire combines dichotomous yes-no questions and rating scales from 1 to 5 (1 being the lowest and 5 the highest). In addition, open questions were used.

Participants´ profile

The first part of the evaluation concerns the characteristics of the sample. The following information provides an overview of the most important data of the group:

• 40,5% (30 persons) of the participants indicated, that they are master students. 27% (20 persons) are enrolled as doctoral students and 16,2% (12 persons) are practitioners in the field of adult education. The remaining 16,2% (12 persons) are either staff at a university, moderators, or were studying another study programme.
• The broad internationality of the Academy is shown by the countries, where students already achieved or will receive their degrees. This year 23 different countries were named, while 8 of the countries are European, the remaining 15 countries are from other continents.
• From everybody answering (N=69) the question of their gender, 78% (54 persons) stated they were female and 22% (15 persons) stated, they were male. No person indicated that they identified themselves as belonging to either one of the other (divers).
• On a scale from 1 to 5, the most frequently named motivational factor for attending the AEA is learning from lecturers from other countries (mean 4,67) and meeting students from other countries (mean 4,57). This is closely followed by the point of improving the own career perspectives (mean 4,4). The motivation for teaching content is not far behind, with “comparative studies” (mean 4,36) attracting greater interest than “European lifelong learning strategies” (mean 4,23). The lowest encouragement for the Academy is in travelling to Germany (mean 3,3) and improving English language skills (mean 3,2).
• Most participants (61%) received a recommendation from their professors to attend the academy. This is especially the case among master's students (83.3%), but also among doctoral students (70%). The lowest figures are found for advertising through flyers, newsletters, and similar advertising media (13.3%).
• The most support from students' home universities on a 5-point scale is for the application process (mean 4.17) and for individual help when needed (mean 4.16). Financial support seems to be the least (mean 3.15).
• For 56,1 % of all the participants the Academy 2023 was the first international study mobility. Looking at the master’s students in particular, the percentage is with 76,7% even higher. The students who have already participated in a study mobility mostly answered that they have participated in an international conference (16.4%).
• When asked, how intensively the students dealt with the topics of the AEA before, a mean of 3,49 was the result. There are no major differences between the different groups, with the highest score being among doctoral students (mean 3.68) and the lowest among practitioners (mean 3.25).
• The question about interest in the various topics shows differences. Adult education in general reveals high values for interest in the topic (84%). Lower values are found for the topic of trans/international and comparative adult education in particular (60%).
• A large proportion of participants estimate having already the competences to interact with people from other cultural backgrounds before the academy (72%). English language competencies were close behind (70.7%), as communication in English is a necessary component of the Academy. The lowest score was in understanding adult learning and education in other countries (30.7%), which leaves room for learning opportunities during AEA. Comparing the understanding of adult learning and education to methodological skills for conducting comparative research, there is a small increase by almost 7% to 37.3%.
• 20% (N=6) of the master students are planning to take up doctoral studies, while 33% (N=10) have not yet decided, what they plan to do in their near future. 30% (N=6) of the doctoral students want to work in an academic or research career exclusively, while 15% (N=3) want to work in a professional career. Almost half of the doctoral students (N=9) don’t want to stay in only one of the two fields and are planning to work in both.

Assessment of the programme

Looking at the overall evaluation of the AEA, a 5-point scale was used for assessment (figure 1). It can be seen that the mean value of the practitioners has a higher total with a mean of 4,7, than the master students with a mean of 4,3. The doctoral students are on the same range with a value of 4,53, compared to the total of all groups, which is 4,56.

Fig. 1: Overall rating of the Adult Education Academy 2023

Next, the evaluation asks the respondents about their specific evaluation of the organizational, academic and didactical quality of the programme modules (figure 2).

Fig. 2: Organisational, academic and didactical quality of the AEA 23’s programme modules

Effects and outcomes of the Adult Education Academy

Effects on the interest in the topics are presented in figure 3. All values have a mean of at least 4,3.

Fig. 3: Effects of the AEA 23 on the participants’ thematic interest

About 81% (N=39) scored either a 4 or a 5 when asked about their motivation for further study mobility, resulting in a mean score of 4.34.

Participants' further study plans were the next question asking masters and practitioners about their motivation for pursuing doctoral studies. Approximately 76.5% (N=26) checked either a 4 or a 5 when rating their motivation for doctoral study. Doctoral students were asked about their motivation for postdoctoral study, with 79% (N=15) ticking that they were highly motivated.

The impact on the improvement of participants' skills is presented in Figure 4. Improved understanding of ALE in other countries (mean 4.64 / N=67) and helping to see ALE from a different perspective in their own country (mean 4.46 / N=67), were highlighted. These two categories scored the highest.

Fig. 4: Effects of the AEA 23 on the participants’ competences

At last, the personal, academic and (only for practitioners) employment outcomes of the participants will be presented. Those categories will be differentiated between master’s students, doctoral students, and practitioners.

• Master’s students: The academic outcome shows a mean of 4,28 (N=29), while the personal outcome is a bit lower with a mean of 4,25 (N=28).
• Doctoral students: Compared to the master’s students, doctoral students show higher results in both categories. The higher results can be seen at the mean of 4,5 (N=18) in the academic and 4,56 (N=18) in the personal category.
• Practitioners: Just like the doctoral students, the practitioners show higher result, than the master’s students. Both, in the academic and personal outcomes a mean of 4,36 (N=11) can be observed. The outcomes for the employment of the practitioners are at a mean of 4,27 (N=11).

Hybrid setting

For the first time, the AEA was held in a hybrid setting. This means, that the participants were both, online and on-site. The first week was held fully online, while the second week was mostly on-site, with the possibility of a virtual participation, for those who were not able to or did not want to travel to Würzburg. The last two years were held online only due to the Covid pandemic, which led to the hybrid setting to ensure the ability to quickly switch to online only if necessary.

This change was also evident in the evaluation. Here, the most common recommendation or suggestion for improving the AEA was the topic of setting. 11 suggestions about the setting refer to the wish for a longer phase on site and at the same time a shorter online phase. Nevertheless, some students mentioned the importance of the hybrid setting and showed their support and suggestions for improvement of this form of learning.

For the next year, both weeks are planned to be on-site, while the possibility of an online participation will still be given.

Evaluation of the Adult Education Academy 2021

External Evaluation: Katrin Riß

INTALL-Project Coordinator: Prof. Dr. Regina Egetenmeyer (Julius-Maximilian University Würzburg)

This outline summarizes the main results of the short-term evaluation of the Adult Education Academy (AEA) 2021 – until 2020 referred to as the “INTALL Winter School”. The respective evaluation step was the fifth out of six within the programme ‘INTALL - International & Comparative Studies in Adult Education & Lifelong Learning’ (2018-2021). Further details are spelled out in the evaluation report.

Since the beginning of our external evaluation in 2019, we continuously see a very positive feedback by the participants and even a rising tendency in most of the findings. All in all, the programme seems to be well geared to its target groups and successfully reaches its goals in fostering interest, knowledge and competences in the field of adult education and lifelong learning. It catches one’s eye that other than in 2020, when master students tended to rate highest, in 2021 the doctoral students seemed to be most satisfied with the AEA. Practitioners obviously were only slightly less content with the programme. Master students close the list of subgroups, still being very pleased with the AEA.

Following we will pick out some findings to illustrate this overall picture. Also, we will summarize some results regarding the effects of the AEA.

The AEA 2021 – programme and evaluation

In 2021, the worldwide coronavirus pandemic took its toll on the AEA. Due to the binding contact and travel restrictions, the programme had to be transferred from a blended learning module to a fully virtual programme. All participants and all staff attended the AEA via their digital devices at home. All lectures and group work (including excursions and socializing events) were realized via video chatting services and further online collaboration tools. Nevertheless, the organizers succeeded to implement the agenda approved during the previous year without any major changes. Some modules were further optimized (following previous evaluation results). For example, the module “Employability in adult education” was expanded and transferred to the online preparation phase.

The external evaluation continued to work with the questionnaire developed by the organizers in previous years. In 2021, the analysis is based on a total of 61 cases.

Participants’ profile

Quite some questions of the short-term evaluation deal with the participants’ profile and their run-up to the AEA. The following findings intend to give an idea of the group.

• At the time of the AEA, 46% of the respondents (28 persons) were studying for a master’s degree, 29% (18 respondents) were doctoral students, and 23% (14 respondents) were practitioners (in the field of adult education and lifelong learning).
• Again, the AEA impressed with its broad internationality. According to the registration data participants of 21 different countries of origin joined the programme, studying in 15 different countries. These included the countries of the INTALL consortium as well as others, a fact that again reveals the internationality of students studying at the INTALL partner universities.
• In general, participants apply for the AEA due to content-related reasons, due to the wish to meet student/lecturers/practitioners from other countries and due to career-related motives equally. Besides, half of the doctoral students (50% / N=18) and around one third of the practitioners (31% / N=13) indicated to have registered in the AEA 2021 because they had attended it before (respectively the predecessor winter schools). This applies to none of the master students. Also, recommendations of former participants seem to be a strong inducement to participate. Among the master and doctoral students this applies for roughly half of the group (master students: 46% / N=26, doctoral students: 56% / N=18). Also, 30% of the practitioners agreed to that (N=13).
• 59% (26 respondents / N=44) (2020: 70% / N=56) of the master and doctoral students indicated that they had received some kind of support by their home universities. 95% of them agreed that this support had been helpful (4 or 5 on the 5-point-scale, mean value 4,77 / STD 0,528 / N=22).
• Almost half of the group had not undertaken any international study mobilities before attending the AEA (49% / N=61). 38% indicated they did so once. Obviously, these shares differ between the subgroups. While for 61% of the master students (N=28) the AEA was their first study mobility, this is true for 44% of the doctoral students (N=18) and for 36% of the practitioners (N=14).
• In 2021, the share of participants who had never or only little been in touch with trans- or international topics in their study or work before was bigger than in 2020. All the same, 70% (N=54) of the respondents agreed they already had been highly interested in trans-/international and comparative adult education/lifelong learning in particular before the AEA. Also, 73% (N=51) had been highly interested in focussing on trans-/international aspects in their further studies and even 81% (N=52) in their future professional activities.
• Further, 70% of the master students (N=23) and 75% of the practitioners (N=8) stated they were highly motivated to take up PhD studies before the AEA. 81% of the doctoral students (N=16) were highly motivated to take up postdoc studies after their graduation.

Assessment of the programme

As mentioned before, the overall assessment of the AEA in 2021 turned out exceptionally well. The mean value at the question “What is your overall evaluation of the AEA?” was 4,68 (STD 0,831 / N=47) (five-point-scale, 1=poor, 5=excellent). The subgroup of doctoral students even rated with the highest possible rating at all (mean value 5,00 / STD 0 / N=15). This general picture is also confirmed by a set of questions asking how satisfied students were with the organizational, academical and didactical quality of the different programme modules. None of the modules achieved mean values lower than 4,20. Doctoral students were most satisfied. 

In the evaluation participants were also consulted, how useful they considered the different components for their own development. Except for one component all mean values lie above 4,00 and mostly range around 4,50 on the five-point-scale. Compared to 2020, most of the results turned out higher.

Effects of the Adult Education Academy

The evaluation also keeps track of the effects and benefits of the AEA as assessed by its participants. In 2021, obviously the strongest effects were achieved with the group of doctoral students, followed by the practitioners and then the group of master students.

All results regarding the effects on the thematic interest in adult education and lifelong learning as well as its international aspects turned out higher than in the previous years. All mean values ranged between 4,31 and 4,56 (N=55) on the 5-point-scale (2020: 4,04 to 4,29, 2019: 4,16 to 4,31). The effects on the motivation to undertake further study mobilities continued to be high. 90% (N=41) of the master and doctoral students ticked a 4 or 5 at the respective question. With regard to further study plans 63% of the master students and practitioners (N=32) stated that the AEA had increased their motivation to pick up doctoral studies (4 or 5 on the 5-point-scale). 87% of the doctoral students (N=15) confirmed this effect with regard to a possible post-doc research after their graduation (4 or 5 on the 5-point-scale).

Effects on the interest in adult education and lifelong learning

Further, the findings on the effects on competences related to international study and work contexts in adult education and lifelong learning resulted very high and exceeded the values of the previous year. Again, the ratings of the doctoral students turned out to be the highest among the subgroups (mean values between 4,31 to 4,75 with a relatively low spread of data points). Little lower we find the results of master students (mean values from 3,54 to 4,27).

Effects - Competences (by subgroups)

Perspectives

All in all the findings drew a very positive picture of the AEA, and it seems that the organizers successfully transferred the programme into an online concept.

In 2021 the AEA’s funding within the project ‘INTALL - International & Comparative Studies in Adult Education & Lifelong Learning’ will come to an end. Also, considering the current pandemic situation, it continues to be uncertain if the AEA – in case of further funding – will return to the face-to-face event in Wuerzburg next year. Nevertheless, before closing this report, we would like to highlight some final aspects worth mentioning or worth considering for future programmes. (1) As in the reports 2019 and 2020, at this point we will pick out the employability module. This module underwent several changes over the last years. In 2021 now, it was extended, split up in several sessions and shifted to the preparation phase. Along with these changes the participants feedback has improved since 2019 remarkably. The results in this year’s evaluation finally are on par with those of the other modules. All subgroups seem to have benefitted more than in the years before. (2) In the past evaluations, respondents often highlighted the connecting and networking effect of the Winter School/AEA. At this of course the informal parts of the programme and the socializing components did their bit, however were hard to compensate within the virtual setting. Nevertheless, the results show that the AEA obviously succeeded in its goal to foster networking among its participants. (3) On the other hand, the virtual setting also implied a number of learning opportunities. Various comments positively referred to the variety of online applications for digital teaching and learning introduced by the AEA. However, some participants also felt that this variety was too vast. (4) Some comments finally suggested to further exploit the chances of the online modus. For example, field visits to practice institutions could be extended to institutions in other countries than just Germany.

Evaluation of the Winter School 2020

Summary of the external evaluation results
Author: Katrin Riß
INTALL-Project Coordinator: Julius-Maximilian University Würzburg, Prof. Dr. Regina Egetenmeyer

The Winter School Programme 2020

In February 2020, the INTALL-Winter School was held for the second time. Compared to the previous year, three main modification were applied, which again proves the programme’s adaptability and the organizers’ constant effort to improve the Winter School according to evaluation results and observations made. First, the heterogeneous interests and needs of the target group of practitioners were taken more into account. By opening both Winter School weeks to the group of practitioners, more options were given especially to those who were interested most in dealing with theoretical approaches of adult education and lifelong learning (week 1). Second, in 2020 the organizers provided a more differentiated and extended programme regarding the topic of employability, a component that stuck out in the 2019 evaluation. And third, responding to the considerable popularity of the Winter School, the organizers increased the number of participants to a total of 83 (during week 2) compared to a total of 67 in 2019.  

Approach, data basis and perspectives of the external evaluation

Some alterations aside, the short-term survey 2020 continued to take a quantitative approach. The survey was based on the very questionnaire used in the evaluation cycle 2019, implemented as an online survey at the end of the Winter School. In total, 70 questionnaires were filled in, which sum up to a response rate of 84%. To give a more detailed picture, in 2020 the study often differentiates between the subgroups of the survey’s population, i.e. the master students, the doctoral students and the practitioners.

Sample/group characteristics

The group of participants was again particularly characterized by its great internationality. The number of countries of the students’ home universities or employers totaled to 16 in 2020, a little more than half of them were European countries (56%). Further the participants backgrounds included 33 different countries of origin and 26 different countries of present residence. This range again far exceeded the consortium of the ten INTALL partners and reflects the considerable internationality of the INTALL partners as well as the remarkable outreach of the programme.

In 2020, a little less than half of the respondents were studying for a master’s degree at the time of the Winter School (31 respondents, 44%). Around one third were PhD students (25 respondents, 36%) and 1 out of 5 belonged to the group of practitioners working in the field of adult education and lifelong learning (20%).

Regarding the gender distribution, 71,4% of the group were female, 28,6% male.

Participants’ Satisfaction

The international Winter School in Wuerzburg has received remarkably good evaluation results during the last years. The Winter School in 2020 is no exception to this. In an overall perspective, the participants rated the Winter School programme with a mean value of 4,33 (SD 0,675) on the 5-point-scale and thus nearly as high as in 2019 (4,39). While the practitioners stayed very steady in their assessment (from 4,11 in 2019 to 4,14 in 2020), the doctoral students rated a little lower (from 4,47 in 2019 to 4,04 in 2020) and the master students a little higher (from 4,42 in 2019 to 4,65 in 2020).

With regard to the organisational, academical and didactical quality of the different programme modules, the results turned out to be rather high. The respective mean values ranged between 3,70 and 4,56. Strongest modules seemed to be the Preparatory Phase and “Lifelong Learning Strategies in Europe/ International Strategies in Adult Education”. At this point we see a rising tendency since 2019. 10 out of 15 mean values (5-point-scale) turned out to be higher than in the previous Winter School.

Also, the participants were asked how useful they considered the different programme components for their own development (rating on a 5-point scale with 1 = not useful and 5 = very useful). Almost all mean values of the components turned out to be higher than the value of 4,00. 23 out 35 items were rated higher than in the 2019.

Effects of the Winter School

The results of the short-term-survey in 2020 equal those of 2019 remarkably. Again, the Winter School had strengthened the participants’ interest in adult education and lifelong learning considerably and also increased their interest and motivation to focus on its trans-/international and comparative aspects in their further studies or future professional activities. All items asked for in the questionnaire received very high mean values between 4,04 and 4,29 on the 5-point scale (1=not at all, 5=very much). Taking a closer look at the subgroups the survey shows that doctoral students tended to rate these questions a little lower.

Also, participating in the programme seemed to increase the respondent’s motivation to undertake further study mobilities considerably. 84% of the master and doctoral students (N=56) scored a 4 or 5 at this question. The respective mean value was 4,32 (STD 1,011).

Further, 66% of the master students and practitioners (N=45) ticked a 4 or 5 when being asked if their motivation to take up PhD studies had grown (2019: 56%). 64% of the doctoral students (N=25) chose a 4 or 5 to indicate that their interest in taking up postdoc research after graduation had increased due to the participation in the Winter School.

And finally, the Winter School also intends to have an effect on competencies related to international or internationally influenced working contexts. In 2020 again, a large share of respondents clearly agreed on these effects. First, the Winter School obviously has a large impact on the development of intercultural competencies. 82% of the participants indicated that the Winter School had helped to improve their “competencies in interacting with people from other cultural backgrounds” considerably by rating a 4 or 5 (mean value 4,21 / STD 1,001). Also, 87% of the respondents stated that their “understanding of adult education and lifelong learning in other countries” had improved by ticking a 4 or 5 (mean value 4,37 / STD 0,745). 74% scored a 4 or 5 when being asked in how far the Winter School had helped to “develop different perspectives on adult education and lifelong learning in their own country” (mean value 4,06 / STD 1,048).

Regarding methodological skills, analytical competencies and professional network competencies the results turn out to be a bit lower. The mean values at the respective items are 3,81 (STD 1,115), 3,78 (STD 1,104) and 3,76 (STD 1,173). The results of the different subgroups at these effects differ considerably. Except for a better understanding of adult education and lifelong learning in general, master students seem to profit most regarding the development of their competencies.

Table 1: Effects - Competencies (by subgroups)

Conclusions

The short-term evaluation of the INTALL-Winter School 2020 again shows that the programme is highly appreciated by its participants and the effects must be considered as high. In an overall perspective the results equal those of the previous year(s) and continue to be on a very high level. Again, the evaluation shows that especially the welcoming, respectful and encouraging learning atmosphere is of great value to the participants. Particularly good and noticeably higher ratings compared to 2019 were given to the module “Lifelong Learning Strategies in Europe/ International Strategies in Adult Education” which is realized during week 1 of the Winter School.

In an overall perspective, it seems that the master students are most satisfied with the Winter School experience. Looking at their motivation we saw that they started with the broadest interest in all aspects and learning opportunities of the programme and also have profited in a very broad and multiple sense. Doctoral students obviously seemed to be focused especially on the content level of the Winter School and in general gave lower ratings than the masters (and mostly practitioners) did. Practitioners finally entered the programme mostly for reasons of exchanging with other experts/fellows in the field as well as looking for expertise in a theoretical but also practical sense, and in the end seem to be fairly satisfied with the Winter School.

The evaluation 2020 produced the following conclusions for further improvement, which however only concern smaller aspects and do not show a need for greater modifications: (1) Since 2019, considerably chances have applied with regard to the practitioners and obviously already have led to various effects. The practitioners’ role seems to be clearer, the integration in the group obviously was improved and all in all we find the practitioners’ satisfaction with the programme to be quite high. Nevertheless, some (single) findings suggest that in week 2 the role of practitioners during the comparative groups could be further developed. (2) The Employability Day has been changed since 2019 considerably and in 2020 the respective evaluation results have improved. Nevertheless, this module seems to bear further developmental potential compared to the other parts of the programme. Unfortunately, the findings give no further hints in which way this component should be altered.

Mid-term Evaluation 2021 - Summary

Author: Katrin Riß
INTALL-Project Coordinator: Julius-Maximilian University Würzburg, Prof. Dr. Regina Egetenmeyer

Approach and data bases

The mid-term survey 2021 was the final step of the external evaluation of the INTALL Adult Education Academy (formerly INTALL Winter School). Based on the concept of the previous years it addressed former participants of the years 2016 to 2020. In 2021, the number of cases was relatively low (N=38) and comprised 15% of the participants addressed, 11% of all former participants. Most of the data of the survey reflect the participants’ perspective one to three years after the participation in the AEA. 39% (N=32) of the data refer to a participation in the previous programme cycle (2020). 21% refer to a participation in 2019 and further 24% reflect the perspective after three years.

Sample/Group characteristics 

Again, the survey succeeded in capturing the broad spectrum of countries the AEA’s participants are related to. In the years 2019 to 2021 the programme had gathered participants from 15 to 20 different countries each year. The mid-term survey 2021 includes 15 different countries where the participants had received or will receive their master or PhD degree. 54% (N=35) of the master’s degrees and 59% (N=17) of the PhD degrees were or will be received at European universities (Germany, Italy, Hungary, Portugal, Norway). Non-European countries represented in the survey are Brazil, Georgia, Ghana, India, Nigeria, Palestine, South Africa, Ukraine, and the USA.

58% (22 persons) of the respondents were still studying when participating in the survey. The exact same share (58% / 22 persons) indicated they were currently employed or had been employed since their participation at the AEA. 10 out of theses 22 were working in the practice or policy field of adult education and lifelong learning, 7 were in the research or academia of adult education and lifelong learning.

Results

The results of the survey 2021 perfectly fit in the picture drawn by the mid-term surveys in 2019 and 2020 and confirm and consolidate the overall picture before.

1. Regarding the effects on the participants’ further studies the data in 2021 was rather scarce. Nevertheless, 14 out of 24 persons (58%) indicated that the AEA had rather large effects on their master or PhD theses (ratings 4 and 5 on the 5-point-scale), a figure that matches the results of the previous year (55%). The mean value is 3,46 (STD 1,215). 64% (ratings 4 or 5, N=22) confirmed they got a more differentiated view on their topic while the same share (64%, N=22) stated that the AEA had encouraged them to include international (-comparative) perspectives in their theses.

2. Looking at future work contexts, 81% (N=26) of the respondents clearly confirmed a motivating effect to work on international issues (ratings 4 and 5). 58% stated that the AEA had provided them with helpful information and contacts for finding future work. Also, 46% chose a 4 or 5 when asked to what extent the AEA gives them a competitive advantage on the labour market.

3. The results concerning the AEA’s effects on the respondents’ competences turned out very clear and confirm the results of 2020. All competences asked for were rated with mean values between 4,19 and 4,59 (N=32). The highest effects obviously are achieved regarding competences in interacting with people from other cultural backgrounds. Here, 94% ticked a 4 or 5 on the 5-point scale. Further, a vast majority highly confirmed effects on their understanding of adult education and lifelong learning in other countries (88% ratings 4 or 5). Networking competences were rated a little higher than in 2020. The mean value rose from 3,98 to 4,28 and thus approached the other competences’ high level.

4. The survey finally intended to learn about the effects on the respondents’ networking. Obviously, the AEA creates a network the participants feel they can rely on. 78% (N=32) confirmed that fact by ticking a 4 or 5 on the 5-point-scale. This obviously not only concerns a professional level. More than half of the group confirmed they had made “really good friends” at the AEA (56% ratings 4 or 5). Regarding a deeper networking process, the opinions differ. Little more than half of the group agreed that they have started a very close exchange or cooperation with some participants (53 ratings 4 or 5).

Mid-term Evaluation 2020 - Summary

Author: Katrin Riß
INTALL-Project Coordinator: Julius-Maximilian University Würzburg, Prof. Dr. Regina Egetenmeyer

Approach

The mid-term survey 2020 again was based on the evaluation concept of the previous years. It follows a mainly quantitative approach realised as an online questionnaire. The call for participation again reached out for all former participants of the Winter School years from 2015 to 2019. In 2020, 49 persons fully completed the questionnaire, around 70% of the sample size of 2019 (71 cases). For the first time the sample also included practitioners, a subgroup that the Winter School was opened for in 2019.

Sample/Group characteristics 

The survey successfully addressed the large international spectrum of Winter School participants, which of course is to be considered as one of the outstanding features of the programme. The number of countries where the respondents had or will earn their degrees (master, PhD and both) amounted to a total of 15. 72,7% of the master’s degree and 56,5% of the PhD degrees were earned at European universities. The list of European countries included Germany, Greece, Hungary, Italy, Norway, Portugal, Romania, Serbia, Slovenia and Spain. Further, participants from India, Malaysia, Morocco, Nigeria and the USA have answered to the organisers request.

At the time of the survey, approximately two out of three participants were still studying or intended to continue their master studies with PhD studies (33 persons, 67,4%). About one out of four (12 persons, 24,5%) on the other hand had finished studies already. In 4 cases the study status remained unclear or was indicated as “other”. Further, 32 persons (65,3%) were or had been employed since their participation in the Winter School. This far exceeds the respective share in 2019 when 19,7% (14 persons) indicated such circumstances.

Results

In summary, the findings of the mid-term evaluation 2020 confirm the results of the previous year(s) and give evidence that the Winter School programme widely succeeds in its aims. These continuously good results surely confirm the concept and realization of the Winter School programme and prove it to be an effective learning opportunity. In detail:

(1) The Winter School obviously had remarkable effects on its participants further studies. 90,9% of the PhD students and PhD graduates agreed that the Winter School had strengthened their motivation to take up PhD studies (ratings 4 or 5 on the five-point scale). Among the  respondents who at the time of the survey were studying for their master’s degree or had finished their master’s degree 62,6% did so. The chart below illustrates that the Winter School also helped its participants to progress their studies and theses in content and methodological related terms. At this, around half of the group (55,3%) agreed that the Winter School had encouraged them to include international and/or international comparative perspectives in their theses, which of course was one of the main interests of the Winter School.

Figure 2: Effects on Thesis

(2) Regarding its work-related aims, the Winter School indeed also strengthened the participants’ motivation to work in international work contexts. At this, the data of 17 persons “currently working” and 18 persons “currently not working” was available. In both groups the majority again highly agreed that the Winter School had strengthened their motivation to work in international work contexts (83,4% / 76,4%, ratings 4 or 5). 38,9% of the participants currently working felt that the Winter School had provided them with useful information for finding their employment. 64,7% of the former participants not working at the time of the survey strongly agreed to that question.

(3) A remarkable number of participants felt that the Winter School had fostered their competencies needed in work contexts including international aspects of one kind or another. These regarded all competencies suggested in the questionnaire including intercultural competencies, knowledge and understanding of adult education and lifelong learning in other countries, English language competencies and to a slightly lower degree analytical and methodological competencies. The chart below illustrates the details.

Figure 3: Competencies - Effects of the Winter School

(4) And finally, the mid-term survey again confirmed that the Winter School successfully provides a (growing) professional network to its participants and induced close exchanges and cooperation among its participants. 75,6% of the respondents felt to be part of a network they could contact whenever needed (ratings 4 or 5). Around half of the group (51%) agreed to have established very close exchange or cooperation with other participants. The use of the LinkedIn-networks related to the Winter School seems to be rather individual. 28,6% of the respondents stated they used these networks in an active way (posting news, participating in discussions), 34,6% follow the news and discussions but contribute only occasionally.