EVALUATION OF GUIDANCE AND COUNSELING PROGRAM DURING THE COVID-19 PANDEMIC USING THE KIRKPATRICK MODEL

This evaluative research was conducted at a private elementary school in Bogor City to evaluate the Guidance and Counseling program during the Covid-19 pandemic using the Kirkpatrick model. Kirkpatrick's four-level evaluation was carried out using questionnaires, interviews, and field notes on students, Guidance and Counseling teachers, classroom teachers, and parents. The results of the reaction evaluation showed that most (56%) students were satisfied with the implementation of the guidance and counseling program. The program changed students' cognitive, affective and psychomotor aspects, with the highest percentage in the affective aspect (49%). Besides that, students also experienced a pretty good change in behavior after participating in the program. In addition, the classroom teachers, Guidance and Counseling teachers, parents, and students stated that the program during the pandemic positively impacted changing student behavior.


INTRODUCTION
In the last three years, life changes have been felt because of the coronavirus, not only in certain places or people but the whole world. Mobility restrictions have a significant impact on all sectors in every country. Education is one of the sectors that has felt the most impact and must undergo many changes. The real change is that the process of administering education, which is usually done face-to-face in schools, must be done from home. The Minister of Education officially issued this rule through the Circular of the Minister of Education and Culture number 4 of 2020 concerning the Implementation of Educational Policies in the Emergency Period for the Spread of Coronavirus Disease . With this sudden change, there are many problems in preparing and implementing the learning process.
Guidance and counseling are important elements of the school. Sugiyo (2016) states that the position of counseling guidance in the curriculum at the education unit level is very strategic and is the heart of self-development activities. Guidance and counseling services assist students in developing their potential and help shape student character, and make them independent. Through guidance and counseling activities, students are expected to develop themselves as optimally as possible so that the goals of education and guidance are achieved, namely the achievement of a healthy and intact personality. Regarding its important role in schools, the existence of Guidance and Counseling is very much needed as long as students undergo the learning process during the pandemic. Pedroso et al. (2022) concluded in their research that during the COVID-19 pandemic, the role of guidance and counseling was highlighted because they realized the importance of providing additional guidance and counseling services that promote mental health.
Applying for guidance and counseling service programs in Indonesia currently uses a comprehensive guidance and counseling pattern. Gysbers & Henderson (2012)

explain that
Comprehensive Counseling Guidance is a program with an organizational framework using the same language harmony with specific configurations planned, sequenced, and coordinated with guidance and counseling activities and services for students, schools, and community/resources. Others and designed to serve all students and parents/guardians in the local school district. In comprehensive guidance and counseling, four interactive program components describe the main activities and responsibilities of the personnel involved in the guidance program, namely: (1) the guidance curriculum or structured classroom activities, organized around the domain of student competence; (2) individual planning, including activities to assist students in monitoring and understanding their growth and development; (3) responsive services, such as information seeking, crisis counseling, and consultation with teachers and parents; and (4) systems support, with activities directed at program management and operations. In its implementation in Indonesia, the first component of the comprehensive guidance program was changed to basic services.
Guidance and counseling services provided by teachers must be programmed. The Guidance and Counseling Program must be made by adjusting students' needs so that they can adapt and overcome their problems during this pandemic. In field conditions, the implementation of counseling guidance during this pandemic also encountered obstacles.
Counseling guidance services which are usually carried out in person, must be done faceto-face or in the form of delivering material through the Learning Management System (LMS) platform. Supriyanto et al. (2020), in their research, found that not all students received support for learning or counseling services at home due to the constraints of limited technological devices, signals, and limited internet access, besides that technological media in counseling services made it difficult for school counselors to understand the problems and feelings of students in the process of alleviating student problems.
Providing guidance and counseling services during this pandemic is challenging for a guidance and counseling teacher. The creativity and innovation of guidance and counseling teachers must be developed so that guidance and counseling services can still be provided and the benefits are felt by students even though there is no face-to-face at school. Such as the Guidance and Counseling program in a private elementary school in Bogor. Some counseling services implemented during the pandemic include classical services, individual/group counseling, group guidance, and collaboration services with parents and outside parties, where guidance and counseling teachers try to use various interesting service methods to still serve students even from a distance. The guidance and counseling teacher forms several guidance groups with topics relevant to each group. So that students still have a place to socialize and chat with their friends in a warm and relaxed atmosphere.
In addition, parents are also involved in various parenting activities and discussion groups.
So that parents can also follow the progress of students at school. Students become more confident and courageous because parents also have a good relationship at school. Disciplinary problems can also be handled because the school and parents work well together. Khasanah et al. (2021), in their research, stated that the role of parental assistance during online learning has a positive impact on student learning development. Guidance and counseling teachers are also present in the quotes sent in the form of an e-wall magazine.
However, in its implementation, there are also some obstacles experienced by guidance and counseling teachers. For example, some parents still find it difficult to cooperate when the guidance and counseling teacher tries to deal with their problematic child. Parenting programs are held based on existing phenomena and are attended by more parents with children who do not have problems. Meanwhile, parents with problem children tend not to have the awareness to participate in parenting programs. Guidance and counseling teachers try to contact parents with problem children through private channels, but the involvement of parents is still not optimal due to busy work reasons.
In order to develop a better program, the results of the previous program evaluation are a reference that cannot be abandoned. For the guidance and counseling program held in schools during the pandemic to see its effectiveness, it is necessary to conduct an evaluation.
Hajaroh (2018)  Kirkpatrick's evaluation model is usually used to evaluate the implementation of training programs. In their book, Kirkpatrick & Kirkpatrick (2008) explain three reasons why training programs need to be evaluated, including 1) The results of the evaluation can tell how to improve the following program; 2) The evaluation results can be used to determine whether a program should be continued or discontinued; 3) The results of the evaluation will help to justify the existence of the training department and its budget Thus, in general, evaluation should be carried out in program implementation to determine the effectiveness of a program and help determine steps to improve it.
Conducting an evaluation, of course, is not done haphazardly. Some steps must be considered. Each evaluation model has its characteristics depending on the focus to be evaluated. The Kirkpatrick evaluation model has four sequences of ways of evaluating programs, and each level has an impact on the next level (Kirkpatrick & Kirkpatrick, 2008).
The four levels are (1) Reaction, this is the initial stage that must be passed in evaluating program implementation. At this stage, measurements are made of how the participants react to the program they are participating. The reaction here can also be said as a measure of customer satisfaction. The future of a program depends on the reactions of the participants. If participants do not react positively, they will likely not be motivated to learn; (2) Learning, at this stage, the extent to which participants change attitudes, increase knowledge and improve skills as a result of participating in the program. Suppose one or more of these changes occur. In that case, there will also be changes in participant behavior; (3) Behavior, it can be interpreted as a measure of the extent to which behavior changes occur because participants follow a program; and (4) Results, this is the final stage of the Kirkpatrick evaluation. Here, the final results that occur because participants follow the program are measured. Results can include increased production, improved quality, reduced costs, increased sales, decreased turnover, and even higher profits.
To evaluate the counseling program during this pandemic, the authors are interested in using the Kirkpatrick model to assess the program's effectiveness in terms of reactions, learning, behavior, and results. This evaluation research was conducted in a private elementary school in Bogor because that school's guidance and counseling program has never been evaluated and is currently preparing for face-to-face learning. Based on the evaluation results, it is hoped that it will be known how effective the program has been during the pandemic so that it can be decided which programs need to be continued and developed.

METHOD
This was evaluative research using the Kirkpatrick model. The evaluation steps are carried out regarding four levels: reaction, learning, behavior, and results. With four levels of evaluation used according to the stages, the answers needed in this research were provided. The research was conducted in a private elementary school in Bogor, with a total of 115 students from 4 th to 6 th grades divided into six classes. At the same time, the sample was taken from two classes of the total population. The total number of the samples was 50 students, five parents, and two classroom teachers. Data were obtained from various sources using questionnaires, interviews, and field notes. All information was then analyzed both qualitatively and quantitatively. The research procedure was carried out with the stages shown in Table 1.

Evaluating Reaction
The implementation of the guidance and counseling program is considered successful if the students are satisfied. Evaluation of reactions to see student satisfaction with the program's implementation was carried out by distributing questionnaires through the google form. The components that are assessed to see student satisfaction are the objectives of the guidance and counseling program, teacher competence, service quality, evaluation and follow-up, and the quality of guidance and counseling infrastructure.
The evaluation results of students' reactions to the guidance and counseling program carried out during the pandemic are depicted as student satisfaction in Figure 1. The bar histogram in Figure 1 showed that students were satisfied with the counseling guidance program held during the pandemic because there were no students who had a low level of satisfaction.

Evaluating Learning
In measuring the effectiveness of the guidance and counseling program implementation, it is necessary to evaluate learning. Learning evaluation involves three aspects: changing attitudes, increasing knowledge, and improving skills in students. This evaluation is carried out by distributing a questionnaire consisting of 17 items via google form to students, as shown in Table 2. Learning Evaluation Questionnaire.

Counselor
• A Do students implement the counseling services that have been received? • What are the results of the services provided by guidance and counseling teachers during the pandemic? • How do students' behavior change after getting the guidance and counseling services that the guidance and counseling teachers have programmed? • What is the response from teachers regarding changes in student attitudes after receiving counseling services?

Peer (student)
• Describe your observations of positive and negative behavioral changes in your friends and yourself during the pandemic! • In your opinion, how have guidance and counseling services contributed to the changes you have experienced?
The evaluation results of student learning from guidance and counseling activities can be seen in Figure 2. The Bar Histogram of Students' Changes. Based on the figure, it can be concluded that with the guidance and counseling program, most students in a private elementary school in Bogor City have cognitive, affective, and psychomotor changes, although they have not reached 50%. Of the three aspects, the most remarkable changes occurred in the affective aspect, with 49%.

Evaluating Behavior
This evaluation stage is carried out by interviewing guidance and counseling teachers and students. In this evaluation section, the researcher uses interview guidelines to get an overview of the impact of the guidance and counseling program implementation on changing student behavior during the pandemic. The interview guidelines are shown in table 3. Interview Guidelines on Behavioral Evaluation.
The results of interviews with guidance and counseling teachers found that most of the students implemented the counseling services that had been provided. The results of guidance and counseling guidance and counseling services are seen in changes in student behavior, such as applying classical guidance materials, maintaining and caring for the body after attending sexuality education classes, recognizing and controlling emotions, and being more concerned with friends. The response from classroom teachers was also good regarding the counseling services provided during the pandemic. The services provided can be a refreshing activity for students, so they do not just struggle with assignments. In the guidance and counseling class, students are free to express themselves, and the class atmosphere is usually lively and fun, indicating that students enjoy it. Some children being counseled intensively also have good progress, although, in some students, there is no significant progress. The group guidance program has become a program that is quite in demand during the pandemic because students have a place to tell stories and socialize with their friends in a relaxed atmosphere.
The results of the researcher's interview with five students found that the changes in behavior they experienced and observed from their friends, such as, there was a tendency to be quieter and somewhat indifferent when learning, often feeling bored, and sometimes did not pay attention when the teacher was teaching. However, they also experienced positive changes such as being more polite in expressing opinions in chat or class discussions, realizing that interactions with friends are valuable and much missed, and being more diligent in exercising to maintain a healthy body. They argue that counseling services also affect behavioral changes that occur in themselves, such as being wiser in using gadgets, managing rest periods, caring for and taking care of the body, respecting and listening to people who are talking, being more concerned with mental health, and not afraid to come for counseling to the guidance and counseling teacher. The group counseling held every afternoon is also a place to chat casually with friends.

Evaluating Result
This evaluation was conducted by interviewing the class teacher and parents. In this evaluation section, the researcher uses interview guidelines to get an overview of the results of changes in student behavior after receiving services from the counseling program during the pandemic. The interview guidelines are shown in Table 5. Interview Guidelines for Results Evaluation.

Subject Question Items
Homeroom teacher • According to your observations, how do students change their behavior after receiving guidance and counseling services? • How does the counseling service impact the students in your class?

Parents
• According to your observations, how does your children's behavior change after receiving counseling services? • How does the counseling service impact your children?
The interviews with homeroom teachers show that guidance and counseling services play a significant role in student change, especially for students with particular problems.
Intensive individual counseling makes students quite good progress, but it takes time.
Students become more controlled in expressing emotions. Students always look forward to the guidance and counseling class because, in the guidance and counseling class, they are free to express themselves by discussing fun things, and the atmosphere is also not tense because there are no assessments and assignments. The classical materials are exciting and provide new insights to students and teachers who are still unfamiliar with mental health materials. Parenting programs held by guidance and counseling help classroom teachers deal with students' problems that require the cooperation of parents.
Meanwhile, the results of interviews with five parents whose children had received intensive counseling services stated that: their children become more emotionally controlled, easier to communicate with, and disciplined, children become more disciplined when participating in distance learning, more accessible to negotiate about gadgets and those who initially refused to join counseling services are now happy to meet guidance and counseling teachers. The changes experienced by students are a process and must be continuously cared for so that children do not return to how they were before being given counseling services.

Evaluating Reaction
Reaction evaluation is conducted to see students' satisfaction with the implementation of the guidance and counseling Program. The results indicate student satisfaction with school guidance and counseling programs. It shows similarities with the research conducted by Ardinata et al. (2020) regarding the analysis of the level of satisfaction of the 7 th graders at Al-Aqso Junior High School of Cluring towards the implementation of counseling guidance services during the Covid-19 pandemic. The research showed that 72% of students were satisfied with the counseling guidance services carried out by guidance and counseling teachers during the Covid-19 pandemic. However, the research subjects were different, namely, junior high school, while this research was conducted at the elementary level. Both showed satisfaction with the guidance and counseling program organized by the school during the Covid-19 pandemic.
Looking at the indicators used as a measure in evaluating this reaction, it shows that student satisfaction with the guidance and counseling program is due to the precise objectives of the guidance and counseling program, the competence of the guidance and counseling teacher, the quality of the guidance and counseling services provided, the evaluation and follow-up provided by the guidance and counseling teacher, and the quality of the guidance and counseling facilities provided by the school.

Evaluating Learning
Learning occurs if there is a change in attitude, increased knowledge, and increased skills. Changes in cognitive, affective, and psychomotor aspects were experienced by students in a private elementary school in Bogor after receiving guidance and counseling services. These results are similar to the research by Mahaly (2021) entitled the effectiveness of the implementation of personal guidance services by counseling guidance teachers. The study also showed changes in the three aspects of the students after attending private tutoring services. It was explained that 56.1% of students understood themselves through personal guidance services, 53% of students could make their own decisions, and 56.1% of students could solve problems. Although the research variable only focuses on personal guidance, this shows that the guidance and counseling program changes students' attitudes, knowledge, and skills.
The results of this study, supported by previous research, show that the existence of a guidance and counseling program held according to the needs of students will positively impact the cognitive, affective, and psychomotor aspects of students.

Evaluating Behavior
Evaluation of behavior related to the implementation of the counseling program during the pandemic is more focused on changing students' behavior towards the benefits of the counseling service program that has been provided. Through the counseling services provided, students were hoped to experience positive changes in their behavior. Pedroso et al. (2022) conducted a study on the challenges faced by guidance and counseling teachers in implementing services during a pandemic, namely personal and professional challenges heavily influenced by technology. However, in practice, the teachers can seek various methods and approaches to provide guidance and counseling services to students efficiently. This is also experienced by a guidance and counseling teacher from a private elementary school in Bogor city, but in practice, the teacher strives to continue to provide classical online guidance, make quotes about mental health, group guidance to chat casually via zoom, etc.
The creative efforts of guidance and counseling teachers in providing guidance and counseling services bring a unique attraction for students to participate in guidance and counseling services. The interest of these students will make an impression on them, impacting changes in their behavior. Thus, it can be concluded that positive changes in students' behavior after participating in the guidance and counseling program are due to the services provided by guidance and counseling teachers being creative and attractive to students.

Evaluating Result
Evaluation of results is related to the final results that occur by students after participating in the guidance and counseling program. Rahima et al. (2022) studied at a private school in Riau regarding the strategy of guidance and counseling teachers in building self-regulation to overcome student burnout. The results showed that the role of guidance and counseling teachers in schools was very effective in building student self-regulation to minimize burnout conditions. The guidance and counseling program makes students more independent, caring, and responsible. Guidance and counseling teachers also collaborated with parents to form student self-regulation. This is in line with the counseling program conducted by the teacher at the private elementary school in Bogor as the subject of this research. Guidance and counseling programs such as classical guidance, group guidance, and parenting programs help students develop emotional maturity and self-regulation.

CONCLUSION
From the results of Kirkpatrick's evaluation research at a private elementary school in Bogor city, it can be concluded that most of the students (56%) were satisfied with the implementation of the counseling program during the pandemic. Holding guidance and counseling programs during the pandemic changed students' cognitive, affective, and psychomotor aspects. The most noticeable change was in the affective aspect (49%). In addition, the students' behavior also experienced a pretty good change after participating in the guidance and counseling program. Moreover, after the implementation of the guidance and counseling program, classroom teachers, guidance and counseling teachers, parents, and students stated that the guidance and counseling program during the pandemic positively influenced changes in student behavior.