NCHF - Post-Fund Reporting - Monitoring and Evaluation Report

• Annex 1 - Outcome Measure Definitions
• Annex 2 – Measure Achievements by Project

Introduction

This report is a key component of the Natural and Cultural Heritage Fund’s Monitoring and Evaluation Framework and outlines what the Fund has achieved against the four outcomes and seven measures.

The purpose of this report is to obtain an overview and understanding of the impact the Fund has had over its lifespan. This will be achieved by:

• Providing a summary of the Fund’s achievements, and
• Reviewing the evidence provided by each grantee against their agreed outcome measures.  These include quantitative (counts, survey results) and qualitative information (photos, videos, a case study).  The latter are presented elsewhere, with the case studies on our website.

The Monitoring & Evaluation framework was published on NatureScot’s website in 2019 as guidance, alongside a template, and examples of how to undertake evaluation work at on-site survey template and residents' survey template.  It was based on the Guidance Document on Monitoring and Evaluation, produced by the European Commission for the European Regional Development Fund (ERDF) Programme.  It also followed the principles in the Scottish Government’s Monitoring and Evaluation Strategy for the 2014-2020 Scottish Operational Programmes.

The purpose of monitoring and evaluation is to provide evidence of the extent to which we have achieved this.  It also allows us to track the management of the funded projects, the inputs, outputs, results, and the impact that these make both individually and cumulatively.  As Lead Partner, NatureScot is expected to assess whether the NCHF Strategic Intervention has produced the desired results.  This report will also inform the contribution made by the Fund to the National Indicators

Fund Outcomes

These four outcomes were what the Fund set out to achieve, and we have done so.  The thirteen funded projects were scattered across the Highlands & Islands of Scotland, and each has made an impact in their local area, 12 of which were rural, and the 13^th pan-area.

At its core, this fund was about increasing tourism sustainably.  The approach taken throughout the fund, and by each of the projects is about creating places that are sustainable, doing it in a sustainable manner, and this links into the focus of the National Tourism Strategy.  Tourism is also a natural capital asset.  It is notable that cultural provisioning took a Covid-related dip – this was seen throughout the tourism industry, but visitor numbers have increased at all our physical sites, and the investment made in them (particularly in parking and toilet facilities) has made those increases more sustainable. 

This EU fund was specifically targeted at the Highlands & Islands – it is known as a “transition area” by the EU and as such benefits from more funding than developed areas (the remainder of Scotland).  The value of the investment was £9.5m of EU funding, which then attracted other investments to increase to £22m.  What we have more than adequately demonstrated is that we can spend money in places that benefit local communities: the input the locals had into the projects was seen as a real positive, and local communities are also enjoying the fruits of the developments with new places to socialise, or to use for community events.

The 11 projects with physical locations provide a new or improved place for visitors and locals alike. Spirit of the Highlands & Islands and The Coast that Shaped the World are both digital-only, though the Tapestry element of the Spirit Project will end up in Inverness Castle, when the latter opens in 2025. Placemaking is the process of creating good quality places. It concerns the environment in which we live; the people that inhabit these spaces; and the quality of life that comes from the interaction of people and their surroundings.  Several of the projects have enhanced the places they were created in, to the benefit of the people who live nearby, as well as to the visitor, who have more places to see, spend, and stay (thus producing an economic benefit too).

NatureScot is also required to report against the contribution the Fund has made to the EU horizontal themes.  Please see this horizontal themes report to find out more about how each of the projects has developed sustainably, looked for equality between men and women, and actively sought to avoid discrimination.

The Fund also sought to promote the Gaelic language.  While we published all our blogs in English and Gaelic, and certain documents dual language, the projects have integrated it from the start (with the exception of the two in the Northern Isles, where the Gaelic tradition never existed). by producing plans on how it would be used.  Dundreggan’s Rewilding Centre, Corrieshalloch Gorge, and Scotland’s Wildlife Discovery Centre all have buildings, rooms, and paths named in Gaelic first, English second.  The rich vein of landscape, biodiversity, and heritage that runs through Gaelic has been used extensively in Uist, Ardnamurchan, and Skye, being woven through interpretation and events.  Storytelling from Coast and Spirit, responsible for gathering more than 2000 stories between them, have many in Gaelic first.

One outcome the Fund did not set out to achieve specifically relates to wellbeing.  More than 1000 volunteers took part in various projects.  Some gained experience in archaeological digs, others in gathering stories from their local communities, and some made friends for life by taking part in the community tapestry project (Spirit of the Highlands & Islands), or building Kilchoan Jetty together, in the latter part of the pandemic.

The whole of this fund is greater than the sum of its parts – the public sector often has artificial distinctions in how we invest money, with NatureScot investing in nature restoration, biodiversity, access, peatland, etc, while Historic Environment Scotland focusses more on historic buildings and archaeology.  Normally we look to the third sector to bridge that gap, but NCHF has done so, enabled by European Regional Development Funding.

The investment has been in a range of projects with some focussing on nature and access, others on history and heritage, and some on both.

 

 

Result Indicators & Report

Although the fund has not just been about jobs, these are what has been agreed to report on with the Managing Authority and European Commission.  Three indicators:

The first two indicators were a simple count of projects.  They have both been fully achieved, with some of the 13 projects contributing to both, and some only to one or the other.

The third indicator is described as a count of the number of sustainable jobs created by the NCHF that exist for at least 24 months after project closure. This third indicator was originally different.  At the start of the Fund, and pre-pandemic, the intention was to achieve an increase in the number of jobs in sustainable tourism in the Highlands & Islands (a 5% increase on the baseline of 26,200 = 27,510; otherwise known as indirect jobs).  This would have meant an increase of 1310 full time equivalent jobs created by the investment the NCHF had made in sustainable tourism.  The effects of the pandemic have meant that any meaningful and attributable data is impossible to ascertain.  A change to the indicator to measure direct jobs was agreed in September 2022. It is measured in full time equivalent status and the agreed target was 81.76 FTE. Due to a misunderstanding in interpretation of the description of the indicator by some grantees, the Fund has actually created 55.21 FTE jobs.  Administratively the EUMIS system does not record this indicator as fully achieved because we were not able to adjust the target after discovering the misunderstanding.  However, these are 55 jobs that would not have been created if the Fund had not existed and represent full achievement of the revised prediction.

Accuracy Analysis

In general, the monitoring data collected by the grantees has been limited.  Sample sizes have often been small, and repetition of collection rare, though there are some notable exceptions to that.  Attributability is high however, so however large or small the datasets, we know that they are directly delivered because of the Fund.  Some grantees finished their projects relatively late in the programme’s lifespan so meaningful data has not yet been collected.  More will be available after the 2024 tourist season and in many cases allow direct comparison with what had gone before.

Outcome Report

The main outcome for the fund was to make tourism in the Highlands & Islands more sustainable.  Through the provision of new services, and the promotion of what is already in our landscapes and communities, we have achieved that.  Tourism in Scotland is worth £4bn to the Scottish Economy, with the natural environment being the top reason to visit.  87% of communities recognise tourism as an important industry and 94% of Scottish residents view it positively.  Since the pandemic, 61% of visitors want to travel more sustainably (numbers from VisitScotland).  As you’ll see in the remainder of this report, each of the thirteen funded projects has contributed to this significantly, and each in their own unique way. 

The projects have made the assets we have, both natural and cultural, more accessible.  Some have created new places.  Some have made improvements to existing places and buildings by adding or restoring paths, making buildings DDA compliant, or revamping existing structures. For others it has been highlighting what was already there but well hidden, or not displayed in the way it could have been.  They have all concentrated on promoting the riches Scotland offers, moving domestic and international visitors away from the honeypot sites to other places, or to visit in other seasons. 

At the core of all these works though, is storytelling.  Human beings are storytellers at heart and each of the projects tell stories of the past or present, and some the future.

The following are highlights from the projects, with the full range of what was done in annex 2, so not every project will feature in every outcome report, despite contributing to it.

Awareness of the distinctive heritage assets of the region is increased, making them more accessible and better understood in ways which are consistent with their sensitive management and sustainable use.

Report

1. Hermaness Hill Path & Welcome Area created a 2km boardwalk and a welcome area.  This has made the National Nature Reserve more accessible to visitors and at the same time has protected the fragile blanket bog and the birds that call the peatland their home.  The welcome area was constructed with interpretation, telling visitors about the nature they were about to see, and also accessible toilet facilities (which are open for longer periods in the year).  Visitor numbers in 2013 were 5000 a year, and in both 2022 and 2023 were c.10,000.
2. Dundreggan’s Rewilding Centre was created to encourage visitors to understand rewilding.  Prior to the centre being built visitor numbers were 1000 a year to the estate.  In the first six months of opening that figure stood at 15,000 and has been 29,000 in the year since opening (April 23-April 24).
3. Scapa Flow Museum originally aimed to have 17,000 visitors a year by year 5.  Within the first year they exceeded that (18,781), thus promoting the heritage and history of this important era of military history to so many more.
4. The Uist Unearthed project has transformed some of Uist’s most amazing archaeological stories, that were previously locked away in academic monographs.  The app has had 4500 downloads (as of 2023).  One of the five sites, Cill Donnain, had no signposting, no visible remains, or recommendations in Uist visitor guides.  It simply wasn’t a visitor attraction, despite its importance in archaeological terms.  It is likely that because of this project Cill Donnain has seen at least a 90% increase in visitors.  68% of survey respondents stated that using the app would make them very likely to visit/spend more in Uist than they normally would.
5. Kilmartin Museum has seen an increase in education service visits from 3000 per year to 9000 per year; visits to the museum increase from 13,000 to 22,500 per year; and glen visitors from 28,000 to 44,000 per year.  The integration of the experience to highlight both the nature and the culture of the unique Kilmartin Glen is an exemplar for this Fund.

Credit: Orkney Isles Council

Credit: Mike Pennington

Credit: Phil Richardson, Archaeology Scotland

Credit: Ewen Wetherspoon, Highlife Highland

 

The quality of the experience and products on offer to visitors to the region are enhanced through the provision of good quality, accessible facilities and/or information relating to heritage assets, in ways which fill gaps in provision, disperse benefits and encourage or attract visitors to discover and explore responsibly.

Report

1. Visitors surveyed in Ardnamurchan rated their knowledge of the area’s heritage as a 1/5 before their visit, but 3/3 afterwards.  The interpretation created by Archaeology Scotland in the area was rated as useful by two thirds.
2. Visitors to the Quiraing, Old Man of Storr, and the Fairy Pools rated the quality of the infrastructure as 8.9/10, with their “propensity to recommend” at 9.3/10.  Both of these statistics reflect positive visitor experience across the broad range of users surveyed.  Anecdotal feedback was that visitors were surprised at how good the paths were for walking in trainers, and plenty of room to pass other visitors.
3. Users of the Coast app were delighted by the stories collected.  Many users shared the social media posts created by the project to their own pages, tagging others who may find them interesting, and adding personal memories.  Because the app is location based, users can delve into the stories where they are visiting, enhancing their experience.  Users commented that they were learning so much, that they were looking forward to reading more, and looking forward to using it when they visited.
4. Uist Unearthed app users were surveyed and 85% of them stated they would recommend using the app or visiting the exhibition to others.  Prior to the project the 2017 VisitScotland survey found there were no stand-out visitor attractions in South Uist.  96% of the app users surveyed by the project agreed that the project provided more things for visitors to see and do.  Visitors to the sites previously would peer at interpretation boards and unmarked mounds, or stone remains.  Now they actively engage with the archaeology, ducking to get through virtual doorways and dodging fires, and then continuing to explore the information using the app after their site visit.
5. Visitors to Corrieshalloch Gorge now stay at the site an average of 2 hours, a massive increase from the previous 20 minutes.  89,000 of them by September 2023, compared to an annual 146,700 in 2019/20.
6. This outcome was the main focus of the Spirit of the Highlands & Islands project, concentrating on encouraging visitors to lesser-known places in the region.  The website created has had over 90,000 views, 1896 itinerary searches, and 1275 location views.  The average dwell time on the website was over 5 minutes.  The project has generated nearly 12,000 exits from the website to sites relating to accommodation, food and drink, things to do, and experiences.  A bank of imagery was created by the project, covering locations and activities mentioned in the stories collected (stories were collected from locals on what they considered to be the “spirit” of the H&I), as well as covering areas of the Highlands & Islands that were low on image stock.  Seasonal photography was undertaken to expand the shoulder season promotion.   These were used in media promotional work to encourage more visitors, both domestic and international.  Themes were around changing visitor perceptions, encouraging responsible tourism, emphasising seasonal and geographical spread.  Audiences were those with an affinity for slow and immersive travel with natural and cultural heritage, interested in sustainable tourism, and communities.

New products or services based on the region’s natural and cultural assets are developed, creating and/or maintaining jobs, helping to sustain local populations, and benefitting local communities. Projects should contribute to sustainable social and economic growth based on sustainable environmental management and productive economic use of natural and cultural heritage assets.

Report

The jobs listed below are those directly employed by the projects. But the additional expenditure by visitors, longer stays, wider travel (to more remote areas) and greater number of visits in the shoulder months all contribute to this target. We do not have a direct measure of these - but the description of the impacts of the projects in the preceding sections gives a flavour of the scale of this.

1. Adopt a Monument, Ardnamurchan:  2 new jobs created.  35 supported during the project.
2. Strathnaver Museum: 4.13 jobs created.
3. Scotland’s Wildlife Discovery Centre: 14 new jobs created.
4. Skye’s Iconic Sites: no permanent jobs were anticipated, though the ongoing management of the three sites employs 6 individuals (not entirely attributable to NCHF, but also the other funding).  Most of the jobs supported during the projects were either Skye- or Highlands-based.
5. The Coast That Shaped the World: no permanent jobs were anticipated.  Jobs supported during the project were dispersed in rural areas across multiple west coast communities, and during COVID years.
6. Uist Unearthed: no permanent jobs were anticipated, but the two main staff have been placed on permanent contracts by their employers as a direct result of the success of this award-winning project.  During the project four jobs were supported (for context, 1.65 jobs in Uist are worth 18 in Inverness), based on the population of the islands).  Five of the contractors were Hebridean, with only one from elsewhere.  Local vendors were used for events, with a considerable proportion of funding kept and distributed in the Hebrides.
7. Kilmartin Museum: 5 jobs created, bringing the total to 14 FTE.  10 new volunteering positions created.
8. Corrieshalloch Gorge: 6.5 jobs created (three permanent, nine seasonal), an increase from 1.5 FTE.  Local supply chains and companies were used in the design and construction of the facilities.  Local produce is sold in the café.
9. Dundreggan Rewilding Centre: 16.7 jobs created, mostly undertaken by people living in the local community.
10. Hermaness Hill Path & Welcome Area: 0.08 job creation.  A small but important contribution of a cleaner for the toilets in the most northerly island of Shetland, where locals hold multiple jobs.  Local contractors were used in construction.
11. Wildlife Watch Abernethy: 1.6 jobs created.  18 jobs supported during the project.
12. Access to Orkney Heritage: 3.15 jobs created.  During the Scapa Flow Museum development £1.9m was spent directly on local contractors, and contractors using local accommodation: 534 bed nights on Hoy and 156 bed nights on Orkney’s mainland.
13. Spirit Journeys: no permanent jobs were anticipated.  14 contracts to produce creative and digital content, with a quarter of these Highland-based.  One of the three project roles was extended into the second phase of the wider Inverness Castle Experience project.

We also measured the perceptions of local people.  Each of the projects had local engagement built into it. 

1. An average rating of 8.9/10 was given by local people on the upgrading of the infrastructure at the Fairy Pools, Quiraing, and Old Man of Storr enhancing the quality of the visitor experience.  They scored 8.4/10 on the consultation about the interpretation of the sites.  8.8/10 agree that the upgrading has brought benefits to the local people, including reduced roadside parking congestion, reduced littering, improved toileting, and reduced site erosion.
2. 47% of Uist Unearthed survey respondents were permanent residents of Uist, and of those 91% reported that their knowledge had improved as a result of the project’s work.  The project engaged with more than 900 members of the community and the project team found many of them became champions of the work.
3. One local to Corrieshalloch Gorge was sceptical of the project at first, but had their opinion reversed completely and was really impressed at the site and the contribution to the quality tourist infrastructure the Highlands needed.
4. The Tapestry project within the Spirit of the Highlands & Islands work engaged 686 stitchers, all local to the area.  Their comments reflected the benefit to the area, they made new friends, and were able to use local business and heritage sites for stitching meetings.  Community groups and local schools were also part of this project.
5. Wildlife Watch Abernethy gave out golden tickets to local children – 50 tickets were used, giving access to 94 adults and 108 children, all from within a few miles of the centre.

 

Aspects of the distinctive natural and/or cultural heritage of the region (such as Gaelic and the region’s music or textiles) are promoted through innovative digital interpretation projects and interpretative media. This should be done in ways which protect the natural and/or cultural heritage assets being promoted, improve accessibility and information for visitors and expand visitor markets.

Report

1. Coast’s website has had 12,000 visits, with their social media reach at 273k.  Two of the top ten stories in the Coast app were Gaelic language.  The project co-ordinated with destination management organisations
2. Over 10,000 people have engaged with Uist Unearthed.  Half through the app, half through the exhibitions, and a significant portion of these attending either workshops or events.  On-site observation of the app being used suggests that couples, groups, and families, particularly with young children, frequently share the app experience on a single phone.  That suggests that well more than 4500 people have engaged in an archaeological experience on Uist.
3. Traffic to the Strathnaver Museum website has increased by 117%, up to more than 45,000 views in 2023/24.
4. NTS have promoted Corrieshalloch Gorge using destination marketing and travel-trade partnerships, which benefits accommodation and tourism providers locally.  It has increased the profile of Wester Ross internationally.
5. 11,000 users of the Spirit of the Highlands & Islands website have used it to plan their next trip to the area.  The content focusses on the spring, autumn and winter seasons, so we can infer that this will positively extend the tourist season, thus boosting the economic impact of tourism on the year-round economy of the region.
6. Many of the projects that had a built element to them have established or increased physical accessibility: Scotland’s Wildlife Discovery Centre, Wildlife Watch Abernethy (Loch Garten Nature Centre), Scapa Flow Museum, Dundreggan Rewilding Centre, Hermaness Hill Path and Welcome Area, Strathnaver Museum, Kilmartin Museum, Corrieshalloch Gorge, and Skye’s Iconic Sites.
7. Many of the mainland Scotland projects as well as the Uist-based project had Gaelic language built in from the start.  Scotland’s Wildlife Discovery Centre, Dundreggan’s Rewilding Centre, and Corrieshalloch’s physical buildings have Gaelic names first, English second.  All the media promoted by the NCHF fund itself was dual language. Except Orkney and Shetland.

Credit: Outdoor Access Trust for Scotland

Credit: Paul Campbell

Credit: RSPB

Credit: Coast Project/West Highland College

Measure Report

A set of seven measures were identified as contributing towards the four outcomes of the Fund.  These are as follows:

Measure Report

No.	Measure Name
Measure 1 (Outcome 3)	Employment generation - the number of jobs created in delivering your project and then sustained by it in the long term (i.e. not just for the construction or build of the project).  This is to be measured by the number of full-time equivalent jobs resulting from your project whether through direct employment or in sustaining jobs elsewhere in the community.
Measure 2 (Outcome 2)	Visitor experiences and perceptions are positive - measure of the number of people who have participated in a natural or cultural heritage experience as a result of your project and how they rated their experience.
Measure 3 (Outcome 1)	Increase in attendance at heritage events or places of culture or nature - measured by the number of adults who have attended or visited a heritage event or place as a result of your project.
Measure 4 (Outcome 3)	Increases in length of tourist season - the extent to which your project helps to extend the tourist season, expand markets, increase footfall or length of stay to one or more remote and rural area(s).
Measure 5 (Outcome 3)	New products and services - the extent to which your project offers new products or increases the availability of services to meet customer needs, demands or expectations.
Measure 6 (Outcome 4)	Increases in opportunities to spend, see and stay - the extent to which your project provides increased opportunity to spend, see and stay in remote and rural areas contributing to economic growth and social sustainability.  This can be measured using visitor survey information on length of stay in the area and estimates of spend.
Measure 7 (Outcome 3)	Improved perceptions of local people - benefits perceived by local people who live in your project’s area such as: the percentage of adults who rate that it is a very good place to live and/or has been improved by your project and the extent to which they have been able to contribute to local decisions.

The first measure relates directly to the main result indicator.  Not all projects had to contribute to every measure.  A further definition of these measures is shown in Annex 1.

The specificity with which grantees reported on these measures has meant that an overall report is not possible, and the content is otherwise covered in the outcome report section.

 

 

 

Next Steps

Because the UK has now left the European Union, the Commission has not asked for any long-term monitoring on the projects.  This would normally have been the case, based on previous ERDF funding programmes.  All the projects have committed to monitoring their impacts for their own purposes, so the work will not go unmonitored.  However, there is no requirement for NatureScot to request these data and contractually we can only do so until 2023.

The original indicator of indirect job creation, and in general the sustainability of the effect these projects have in their locales, is going to be looked at again.  A Masters’ student from SRUC in Edinburgh is looking at this topic (at our request) for her dissertation in 2024.  Results will be published in due course.

Case Studies for each of the projects were produced as part of their closure requirements.  These can be seen on our website at our NCHF case studies page. 

We have also published a film of the work we have done.

Report Author

Kirsten Makins, Natural & Cultural Heritage Fund Project Manager, May 2024

Contact: [email protected]