Terms of Reference
Independent External Evaluation
Project Title: Strengthening Village Capacity in Tourism Development Based on Local Natural and Cultural Resources (Memperkuat Desa Dalam Mengembangkan Pariwisata Berbasis Sumberdaya Alam dan Budaya Setempat)
Funder: William & Lily Foundation
Project Period: December 2018 – December 2020
Evaluation Period: October 2020 – January 2021
Introduction & Rationale for Evaluation
The William & Lily Foundation (WLF) is an Indonesian-based philanthropic foundation established in 2009 as an outgrowth of the philanthropic legacy of William and Lily Soeryadjaya. As an active grant-making foundation, WLF works closely with implementation partners on four focus areas: education, health care, economic empowerment and enabling environment. WLF is committed to impactful and responsible interventions in its efforts to empower underprivileged communities across Indonesia. Although the scope of WLF philanthropic endeavours is countrywide, its current geographical focus centres on eastern Indonesia, particularly on the island of Sumba.
In December 2018, WLF formed a partnership with the Institute of Research and Empowerment (IRE), in implementing a two-year program entitled Strengthening Village Capacity in Tourism Development Based on Local Natural and Cultural Resources in Sumba Barat Daya. This project is called Desa Wisata. The project aims to develop village-based tourism in two pilot communities by achieving the following outcomes: 1) improving local government policies and programs to support village tourism development; 2) improving village governance to better facilitate village-based tourism; 3) improving partnership between local government and villages; and 4) improving skills and capacity of the communities to develop and manage village-based tourism. Desa Wisata project’s approaches include policy advocacy at district level, strengthening communication forum for tourism actors, village assistance to improve village governance in supporting tourism villages and strengthening the capacity of local organization and BUM Desa in managing tourism village. Project’s beneficiaries are the local organization, BUM Desa, and communities in the respective village. The project also collaborates intensively with key stakeholders, such as village government, civil society organizations and government institutions at district level (i.e. Tourism office, Village Community Empowerment Office and Development Planning Agency at Sub-National Level).
As this project is nearing completion, WLF plans to conduct a comprehensive final evaluation that will provide a 360 degree outlook on –not only project results towards its intended outcomes, but will also evaluate the effectiveness of the management and implementation process of the project, the appropriateness of chosen interventions, as well as to observe whether this project has applied the value for money principal. As per WLF’s policy, this project is subject to external independent evaluation. WLF is therefore seeking individual or a team of evaluators to conduct this evaluation.
Purpose, Scope and Users of Evaluation
The objective of this independent evaluation is to provide a comprehensive evaluation on the project management and implementation process performed by the implementing partner, in this case the Institute for Research and Empowerment (IRE); particularly on how the project was able to meet the agreed upon project objectives, outcomes and outputs, as well as to evaluate whether the project has been carried out in a cost-efficient manner. This evaluation is also aimed at observing the challenges faced and document lessons learned and best practices produced. Furthermore, this evaluation also aims to understand the efficiency and effectiveness of the grants and technical support WLF provided to the implementing partner during the lifetime of the project.
The evaluation will cover the project's various components (such as the expected outcomes, outputs and activities) as reflected in the project document as well as subsequent modification and alterations made during its implementation. The main user of findings and recommendations from this evaluation will be the William and Lily Foundation. Other potential users may include but not limited to the Foundation’s implementing partners and relevant stakeholders as necessary.
Methodology
The evaluation will be based on a participatory approach, involving a wide range of selected key Stakeholders. The evaluation will use both primary and secondary data sources for probing the progress and for generating evidence in support of the findings. Overall, the evaluation approach should refer to OECD-DAC criteria (OECD-DAC recommend standard for evaluation criteria, such as relevance, coherence, effectiveness, efficiency, sustainability and impact). To the extent possible, quantitative and qualitative data will be collected, validated, triangulated and analysed. The evaluation process will include desk reviews, interviews, and observation. The proposed methodology is suggestive and can be adjusted by the Evaluator if considered necessary, in accordance with the scope and purpose of the evaluation and in consultation with the WLF team.
The Project will be evaluated against criteria such as its relevance to organization mission and objectives of organization, the validity of project design, project effectiveness, the efficiency of resource use, the impact orientation and sustainability. To summarize, the external independent evaluation should address the following key questions:
1. Relevance and value to organization strategy and mission statement
· To what extent does the project design take into account the needs of beneficiaries and stakeholders?
· To what extent does the project design align with WLF’s mission and objectives?
2. Validity and relevance of design
· Are the planned project objective, outcomes, outputs and activities aligned?
3. Project achievements and effectiveness
· To what extent has the project achieved their stated objectives?
· What are the main factors –internal to the project and external- that have hindered the project’s ability to reach the desired results thus far?
4. Efficiency of resource use
· To what extent have technical and administrative support been sufficient and adequate to meet project objectives?
5. Effectiveness of grant management; and
· To what extent have the WLF grant management support been adequate to meet project objectives?
6. Impact orientation and sustainability in relevance to the target community
· To what extent are planned results of the project likely to be sustained and/or scaled up and replicated by stakeholders?
· What further concrete steps could be taken to increase the sustainability of the results?
Evaluator’s Responsibilities and Deliverable
1. Key Responsibilities
· The design, planning and implementation of the evaluation and the write-up of the evaluation report, using an approach agreed by WLF and for delivering in accordance with the WLF’s specifications and timeline
· Consulting and liaising, as required, with WLF, stakeholders and partners to ensure satisfactory delivery of all deliverables
· Making herself/himself available, if required, to take part in briefings and discussions, online or other venue, on dates to be agreed
2. Key Deliverables
· Inception Report with Methodology
The inception report should detail the evaluator’s understanding of what is being evaluated and why, showing how each evaluation question will be answered by way of: proposed methods; proposed sources of data; and data collection procedures. The inception report should also include an evaluation framework matrix, a stakeholder list, proposed interview tools/guides, proposed schedule of tasks, activities, deliverables, and outline of final report
· Validation Workshop
To be conducted to validate the initial finding with the respondent and key stakeholders. The activity will be carried out after the completion of data collection. Results of this workshop will be used as additional source of information during analysis and report writing
· Draft of Evaluation Report
The Evaluator will submit draft of evaluation report to WLF. The recommendations, lessons learned, and emerging good practices should be documented in the draft evaluation report
· Presentation of Draft Report
A presentation should be prepared for the WLF on the draft report, to be used during the debriefing
· Final Evaluation Report
To be submitted to WLF as per report outline and timeline. The quality of the report will be determined based on quality standards defined by WLF. The report should also, as appropriate, include specific and detailed recommendations by the Evaluator based on the analysis of information obtained. The report should also include a specific section on lessons learned and good practices that are relevant for better performance and delivery for future project or could be replicated or those that should be avoided
Proposed Time Line
The evaluation is foreseen to be undertaken in the time period of October 2020 to January 2021 with the aim to submit the final evaluation report to WLF no later than mid-January 2021.
Proposal Submission Criteria
The following will be considered minimum contents of the proposal. Please submit in the order listed:
Proposed methodology and tools for the evaluation design
Proposed team composition (clearly stating roles and responsibilities of each team member)
Financial proposal to conduct the evaluation, complete with breakdown of man-hours or days required to conduct each stage of the preparation, data collection tools development, data collection and analysis, field visit/travel, report writing and presentation
Most current Curriculum Vitae, highlighting related experiences and skills to the required scope of work
Sample of previous evaluation reports
Essential Requirements:
A bachelor’s degree in political sciences, international relations, development economics or any relevant social sciences. Master’s degree is preferred
A minimum of 5 years of experience conducting monitoring and evaluation of development programs in impoverished regions in Indonesia, particularly in carrying rigorous evaluation methodology
Demonstrable research and strong analytical skills
Demonstrable strong knowledge on village governance, Village Law (Undang-Undang Desa) and village-related tourism (especially on the newly stipulated District Regulation on Tourism Village Program in Sumba Barat Daya).
Experience in workshop facilitation
Strong interpersonal skills in a multicultural setting, experienced in engaging community leaders and key government officials at subnational levels.
Proposals must be sent to support@wlf.or.id by 27 September 2020 at 05:00 PM Jakarta time at the latest.