An Indicator-based Approach to Measuring Sustainable Urban Regeneration Performance-Part 2, Empirical Evaluation and Case-study Analysis

Urban Studies, Vol. 41, No.4, 757–772, April 2004 An Indicator-based Approach to Measuring Sustainable Urban Regeneration Performance: Part 2, Empirical Evaluation and Case-study Analysis Lesley Hemphill, Stanley McGreal and Jim Berry [Paper first received, January 2003; in final form, October 2003] Summary. This paper examines the sustainability of current urban regeneration practice, through the application of weighted indicators and a points scoring framework. The analysis applies the hierarchical model discussed in the preceding paper of this journal issue to case studies of waterfront areas and cultural quarters in three European cities: Belfast, Dublin and Barcelona. The evaluation permits performance comparisons to be made between the case studies regarding the sustainability of regeneration areas and projects, variations on an indicator set basis and the sensitivity of scores. Conclusions are drawn concerning regeneration practice, the extent to which sustainability principles are adhered to, potential policy benefits and the applicability of the model. Introduction Sustainable development and urban regener- sustainable development, the role of urban ation have become increasingly recognised regeneration in fulfilling sustainability objec- as complementary goals, but there remains tives remains in its infancy. only limited evidence of performance evalua- Sustainable Development: The UK Strat- tions that investigate the extent to which egy (DOE, 1994) recognised the importance regeneration projects have achieved sustain- of urban regeneration in contributing to sus- able outcomes (Carley and Kirk, 1998; tainability by developing areas in an efficient DETR, 1998a; OECD, 2000). Likewise, the way whilst making them more attractive dissemination of what constitutes ‘good sus- places in which to live and work. In a similar tainability practice’ as discussed in the ac- vein, the Urban Task Force in its “Towards companying paper and how this can be an Urban Renaissance” report (DETR, benchmarked remains underinvestigated. 1999b) advocated the need for a What is becoming increasingly obvious is the “comprehensive package of regeneration prominent place given in UK policy to the measures to address both the physical regen- issue of sustainability and ultimately how eration of an area and the economic and this can be best achieved (DETR, 1998b, social needs of the local population” (p. 133). 1998c, 1998d, 1999a, 1999b, 2000). How- This is further supported by the UK govern- ever, despite this policy drive to instigate ment’s Urban White Paper Our Towns and Lesley Hemphill, Stanley McGreal and Jim Berry are in the Centre for Research on Property and Planning, School of the Built Environment, University of Ulster, Shore Road, Newtownabbey, County Antrim, Northern Ireland, BT37 0QB. Fax: 028 9036 6826. E-mail: [email protected]; [email protected] and [email protected]. 0042-0980 Print/1360-063X On-line/04/040757–16  2004 The Editors of Urban Studies DOI: 10.1080/0042098042000194098 758 LESLEY HEMPHILL ET AL. Cities: The Future—Delivering an Urban gathering data appropriate for the indicators. Renaissance (DETR, 2000) which identified Expert opinion plays a major part in the bringing brownfield land and buildings back empirical investigation which seeks to assess into constructive use as a key objective to the measurement of urban regeneration prac- exploit their potential in contributing to the tice in light of sustainability principles. A quality of urban life. The Sustainable Devel- detailed performance analysis of six indepen- opment Strategy for the UK: A Better Quality dent case studies is presented utilising the 52 of Life (DETR, 1999a) lists regeneration as indicators discussed in the accompanying pa- one of a number of aspects contributing to per. The performance of each case-study area sustainable development whilst indicating is compared for each of the five indicator that building sustainable communities re- groups in the model and against an overall mains a priority for government action. How- aggregated score to provide an indication of ever, Imrie and Thomas (1995) consider that relative performance. Sensitivity analysis is the proliferation of new urban policy initia- applied to each indicator to assess different tives since the early 1980s has not been options. The paper provides the basis for the matched by systematic monitoring and evalu- use of the model for evaluation and policy ation. Likewise, Turok (1989) states that the purposes, notably in the identification of the scope and form of evaluative research have issues where present urban regeneration often been illogical, weak and increasingly practice is not fully embracing the sustain- concerned with a narrow range of quantitat- ability agenda. Conclusions are drawn on the ive evaluative criteria. Such a focus on the respective case-study performances and the quantitative measurement of policy impacts successes or shortcomings of present regen- has failed to recognise the importance of the eration policy in creating sustainable out- subjective, experiential ‘life-world’ of human comes, potential policy benefits and well-being (Burns, 2000). Indeed, it is this applicability of the model. failure to appreciate the human aspects of urban policy through the use of purely quan- titative methods of evaluation that has often Sustainable Regeneration Performance led to the disquiet expressed over urban pol- Benchmarking icy evaluation. Hakim (2000) argues that too often the The subject of measuring sustainability per- secondary analysis of an existing data-set is formance has been assessed in detail in the proposed at the expense of qualitative re- accompanying paper; however, an equally search. Similarly Wong (2002) points to- important aspect of measurement is how to wards the need for more qualitative research benchmark this performance. Benchmarking to measure satisfactorily intangible issues, can be considered an effective means of such as community identity and institutional helping to deliver better services by compar- capacity. Perhaps of greater significance is ing performance against an accepted norm the potential to combine both quantitative and learning from other areas/organisations and qualitative techniques within any given (Audit Commission, 2000). This is all the evaluation to ensure that their respective more evident in today’s society with headline strengths can be capitalised on. Indeed, qual- (DETR, 1999a) and best value (ODPM, itative research can be used in combination 2002) indicators providing two examples of with virtually all other types of study, in- circumstances where indicators have been forming the interpretation of more imper- used to measure and subsequently bench- sonal statistical data (Hakim, 2000). mark performance. Benchmarking enables The research reported in this article builds the comparison of quantitative performance upon the concepts developed in the preced- data, identifies how performance differs from ing paper in this issue and draws upon a other areas, how it has changed over time combination of techniques as a means of and whether it can act as the spur for im-  MEASURING URBAN REGENERATION, PART 2 759 provement or an example of how to achieve and work (21.5 per cent); Resource use (17.5 ‘good practice’ (Audit Commission, 2000). per cent); Buildings and land use (18.9 per Practitioners need to be able to establish cent); Transport and mobility (22.1 per cent); the baseline economic and social conditions Community benefits (20 per cent). Further against which they can measure the progress discussion of the indicators, the points scor- and improvement resulting from regeneration ing framework and how they were selected projects (Bristow, 1999). Likewise, it is ac- and tested is provided in the accompanying knowledged that local authorities and devel- paper. opers require practical tools to measure the sustainability of developments (both build- Case-study Selection ings and infrastructure) at site or estate level (Brownhill and Rao, 2002) to ensure that In testing the validity of the model and seek- they meet planning requirements and contrib- ing to achieve a meaningful set of results, ute to wider sustainability objectives. Simi- rigorous selection criteria were drawn up to larly, a crucial part of benchmarking and the ensure that the same basic characteristics use of indicators is to help guide appropriate were present in each case-study area thereby action through the setting of suitable targets enabling a full comparative evaluation be- (European Commission DGXI, 1996). The tween individual schemes. These criteria in- Audit Commission (2000) identifies the key clude: previous track record of the case-study elements of the benchmarking process— area, the hybrid of contacts available, indi- namely, select service/project, identify re- cator coverage, completed schemes, size and sources, identify partners, define and collect scale of scheme, and the occurrence of wa- measures, compare performance, find best terfront and cultural regeneration projects practice/targets, plan change, implement and within prospective locations. monitor outcomes. The selection of the most appropriate case- This paper adopts a similar benchmarking study locations, as governed by these criteria, strategy to the measurement of sustainable identified a number of potential urban regen- urban regeneration and utilises the model eration schemes embracing ‘good practice’. developed and discussed in this journal issue. However, the requirement for differing Although the points scoring parameters schemes including both cultural and water- reflect current legislative or policy standards, front examples reduced the range of possibil- it is important to stress the dynamic charac- ities. A further consideration was the number teristics of the model and its flexibility to of case-study areas. Literature sources sug- permit change as and when new technology gest that the significance and robustness of or information renders current standards as research findings increases with the number insufficient to meet sustainability require- of sites in which a survey is conducted, ments. although the largest single gain occurs when the number of sites is increased from one to two (Sudman, 1976). This was supported by Evaluative Model Overview Yin (1984) who indicated that survey results The preceding paper provides a detailed re- can be strong with a small number of case sumé of the methodology underpinning the studies provided that they are carefully se- research identifying a four-phase process lected. Based upon these considerations, (Figure 1). This paper deals primarily with three case-study cities were selected with the application of the evaluative model at each providing two individual studies to case-study level. In particular, the focus is reflect the waterfront and cultural themes upon phase four—the case-study application underpinning the ‘designated regeneration and sensitivity analysis. The model is based area’. It is considered that the six case studies on 52 individual indicators which have been provided a sufficient and diverse range of weighted into five main groups: Economy examples to allow the indicators to be tested. 760 LESLEY HEMPHILL ET AL. Figure 1. Methodological approach. Belfast provides examples of mainstream Cathedral Quarter, formerly Northside, on UK-based initiatives including the develop- the northern city-centre fringe of Belfast, ment corporation model, Laganside Corpor- although possessing many key physical at- ation, the last remaining development tributes, has been a difficult area to regener- corporation excluding the recent revival of ate with many different initiatives (Berry and the model in Thames Gateway and extensive McGreal, 1993) the latest of which focuses use of gap-funding through urban develop- upon its ethos as a cultural location. ment grant. In Belfast, two key schemes The second case-study city, Dublin, pro- Laganside and Cathedral Quarter fit the vides many contrasts with Belfast from a wider selection criteria of waterfront and cul- policy perspective (Berry and McGreal, tural quarters respectively (Table 1). The for- 1992). Both of the selected schemes in mer has a proven national and international Dublin (Table 2), the Docklands area and the track record. Laganside schemes received cultural quarter of Temple Bar have recognition from the RICS in 1998 by win- benefited significantly from the use of tax- ning their ‘Reclamation of Contaminated ation breaks. Temple Bar is widely accepted Land Award’ and were the subject of an as being a good model of cultural regener- OECD evaluation study (OECD, 2000). The ation with the area promoted as the flagship  MEASURING URBAN REGENERATION, PART 2 761 Table 1. Belfast case studies: summary Laganside—Waterfront Cathedral Quarter—Cultural 140 ha site situated 1 km from Belfast city centre 12.1 ha site located on the northern fringe of the city centre Area first designated in 1987 with the scheme Area designated in 1998 with the scheme pro- details highlighted in the Laganside Concept posals highlighted in the Cathedral Quarter Re- Plan generation Strategy (previously Northside) Area falls under the control of the Laganside Area falls under the control of the Laganside Corporation, a form of development corporation Corporation, but treated as a distinct area separ- set up to oversee the development of the area ate from the main Laganside site Area consists of 6 key commercial sites and 6 Area (will) consist of 2 apartment developments; apartment developments, as well as environmen- a hotel; various cultural and arts developments; tal improvements to River Lagan and the intro- building refurbishments; a multi-storey car park; duction of a cross-harbour road and rail bridge environmental improvements to streetscape Table 2. Dublin case studies: summary Dublin Docklands—Waterfront Temple Bar—Cultural 526 ha site located in the northern area of the 12 ha site located in the heart of Dublin city traditional dockland; 85 ha available for develop- centre ment and a further 10 ha for open space Area first designated in 1986 although an official Area designated in 1991 with development car- Master Plan was not adopted until 1997 ried out in line with the Temple Bar Develop- ment Programme 1996 Area falls under the control of the Dublin Dock- Area fell under the control of 2 independent lands Development Authority (DDDA) which re- companies—Temple Bar Renewal Ltd (respon- placed the Custom House Docks Development sible for deciding funding applications) and Authority (CHDDA) in 1997 Temple Bar Properties Ltd (responsible for car- rying out the development work) Area consists of 6 main development areas each Development carried out in 2 main phases: of which have various objectives ranging from 1991–96 and 1996–1999. Area consists of an promoting residential use, developing landscap- Irish Film Centre, infrastructural improvements, ing, promoting public transport, promoting apartment developments, retail lettings, building tourism/hotel facilities, protecting natural habi- refurbishments, provision of innovative archaeo- tats, supporting cultural development and the logical component, promotion of area as a cul- expansion of enterprise uses tural quarter project to mark Dublin’s year as European approach taken to the stimulation of private City of Culture in 1991 (Montgomery, 1995). investment between tax-based incentives and In a similar vein, the Docklands area has grant-based mechanisms as applied in been widely advocated as a success (Gahan, Belfast. 1993) and long-term sustainability is one of The selection of Barcelona (Spain) widens the objectives of the Dublin Docklands Mas- the research to a European perspective, facil- ter Plan making the area an ideal case study itating comparison with the UK and Ireland in which to apply the indicators (DDDA, dimension. Barcelona also provided two very 1997). In developing the contrast to Belfast, different regeneration schemes—namely, the the Dublin case studies provided the oppor- Ciutat Vella (Old City) as the cultural exam- tunity to compare and contrast the differing ple and the Olympic Village as the water- 762 LESLEY HEMPHILL ET AL. Table 3. Barcelona case studies: summary Olympic Village—Waterfront Ciutat Vella—Cultural 47 ha site located between the Cuitadella Park 386 ha site located in the old historical centre of and the centre of the Poblenou area—part of a the city and constitutes one of the 10 districts zone of coastal facilities (130 ha) into which Barcelona is divided Decision to regenerate the area coincided with Area designated as an Area of Integrated Re- the announcement that Barcelona was to be the habilitation (AIR) in 1986 with 4 special plans venue for the 1992 Olympic Games. In 1989, for internal reform (PERI) to cover the key areas various changes were made to the Plan General in the old city: El Ravel, Casc Antic, Barceloneta to allow a change in the function of the area and Barri Gotic Area fell under the control of an ad hoc body Area fell under the control of a public–private called Holding Olimpico SA (HOLSA) who company called Promocio Ciutat Vella SA were responsible for co-ordinating the develop- (PROCIVESA) to carry out or co-ordinate the ments main works envisaged in the special plans Area consists of various 5-storey apartment The main elements of PERI plans included land blocks with commercial and retail facilities be- clearance, redevelopment or refurbishment of low, the twin Olympic Towers (Hotel and property, creation of public open space and the Office); and high-quality public space provision of modern infrastructure front example (Table 3). The decision to use tors and include: government departments/ Barcelona as a case-study location was based agencies and other development organ- on a number of counts, but primarily the isations, literature/official reports, other re- international reputation that the city has search, end-user surveys of occupiers/ achieved since the Olympic Games. Indeed, employers, consultation with a range of the Urban Task Force report (DETR, 1999b) professionals in the built environment charted some of the regeneration work un- (architects/property developers/estate agents), dertaken as being a good example of the way questionnaire surveys (employees/residents) forward for future practice. Furthermore, the and site survey and investigation. urban design and architectural quality of The five separate indicator sets—namely, what has been developed have been docu- Economy and work, Resource use, Buildings mented in journals, culminating in the RIBA and land use, Transport and mobility and awarding the area with a gold medal for the Community benefits for each case study were outstanding quality of the transformations evaluated on both an individual and collec- (DETR, 1999b). tive basis in accordance with the points scor- ing criteria and rationale highlighted in the associated paper. The case-study results are Case-study Results considered under three headings: indicator Accessing information to enable each of the set analysis, benchmark analysis and sensi- indicators to be assessed required the utilis- tivity analysis. ation of a combination of highly varied meth- ods, applied consistently across each of the Indicator Set Analysis case studies. Some provided hard quantifiable data that could be directly ap- This analysis initially reviews the scoring on plied to the points scoring framework, whilst an indicator basis, followed by the overall others provided more qualitative information points for each indicator group, supple- requiring interpretation prior to the allocation mented by a brief overview of the relative of points. The sources of information utilised performance of the individual case studies in reflect the complexity and variety of indica- delivering sustainable regeneration out-  MEASURING URBAN REGENERATION, PART 2 763 Table 4. Economy and work scoring performance Economy and work indicators LS CQ DD TB OV CV Number of jobs created per 1000 square metres 8 10 6 4 4 2 Net jobs created 6 4 6 4 8 8 Percentage of new enterprises still operating after 3 10 8 10 6 10 8 years Quality of jobs created 4 10 4 8 6 8 Leverage ratios 4 4 4 4 2 2 Performance of incentive mechanisms 6 1 6 2 6 8 Partnership structure performance 6 5 8 5 8 8 Effectiveness of exit strategy 7 5 8 2 8 7 Incorporation of training programmes 8 7 7 7 6 6 End-user scheme satisfaction 8 5 6 6 8 7 Total score 67 59 65 48 66 64 Percentage score 67 59 65 48 66 64 Weighted score (weighting factor 21.5) 1440.5 1268.5 1397.5 1032.0 1419.0 1376.0 Ranking 1 5 3 6 2 4 comes. Derivation of these indicators has cess in terms of the number of jobs created been discussed in detail in the accompanying and the quality of those jobs with both indi- paper. Their richness stems from the number cators scoring highly. However, the rela- and comprehensive range of indicators em- tively poor performance of the incentive ployed across the five component areas and mechanisms indicator, within the Cathedral the extensive nature of the evidence. Quarter and Temple Bar areas, seems to indi- cate that incentives had little influence on the decision of businesses to move to these loca- Economy and work. This constitutes a subset tions. This also reflects a high number of in of 10 indicators formulated to evaluate the situ end-users. From the sustainability per- performance of economic strategies as well spective, the findings of the Economy and as the tangible outcomes of the investment work indicators show that in most of the process in terms of the employment gener- regeneration schemes partnership structures, ated. For this indicator set, the three water- exit strategies and training programmes are front localities of Laganside Belfast, the important, although the low scores awarded Olympic Village Barcelona and Dublin to the leverage ratios suggest a continuing Docklands achieved the highest scores dependency on public-sector money to se- reflecting the economic emphasis initially cure long-term success. needed in these areas to stimulate investment (Table 4). Of the individual indicators, the percentage of new enterprises still operating Resource use. The Resource use indicators after 3 years showed the best performance relate to the effective use of the physical across all case-study areas, supporting the resources whilst dealing with environmental need for longevity of businesses in the long- issues such as the reclamation of materials, term sustainability of regeneration schemes. waste minimisation, energy efficiency and The 2 Barcelona case studies are the only conservation. This indicator group produced examples to embrace significantly the indi- a similar set of results across 5 of the case- cator relating to the percentage of net jobs study areas with only 7 per cent covering the created: both provide over 30 per cent of top 5 performers (Table 5). The Barcelona their workforce from their local area. The case studies lead on these indicators with the Cathedral Quarter Belfast had notable suc- Olympic Village scoring 64 per cent and the 764 LESLEY HEMPHILL ET AL. Table 5. Resource use scoring performance Resource use indicators LS CQ DD TB OV CV Reclamation of building materials 2 2 2 6 2 8 Retention of environmental features 6 6 4 2 2 6 Household waste disposal 2 2 2 2 6 6 Firms undertaking waste minimisation 10 8 10 10 10 8 Energy efficiency: building lay-out/design 8 7 8 9 8 5 Energy efficiency: building materials/construction 6 4 8 8 6 5 Conservation of built heritage 6 10 2 10 6 10 Incorporation of environmental design 4 4 10 6 10 4 Performance of environmental management 8 6 5 1 8 5 Total score 52 49 51 54 58 57 Percentage score 58 54 57 60 64 63 Weighted score (weighting factor 17.5) 910.0 857.5 892.5 945.0 1015.0 997.5 Ranking 4 6 5 3 1 2 Ciutat Vella 63 per cent. The Cathedral regeneration schemes upon property and de- Quarter Belfast provided the poorest per- velopment outcomes. The top-ranked loca- formance in this group (54 per cent) tions are the Olympic Village (78 per cent), reflecting cost constraints involved in im- followed by Ciutat Vella (74 per cent) with proving the energy efficiency of older build- Laganside, Cathedral Quarter and Dublin ings. All case studies scored well in the Docklands all scoring 73 per cent (Table 6). waste minimisation indicator demonstrating These results place the Barcelona case stud- a very high percentage of firms actively ies collectively as the best performers in this undertaking waste audits. However, this sector corresponding with their well-recog- positive treatment of waste did not extend to nised position at the forefront of urban de- household waste disposal, with the Barcelona sign and the creation of buildings with case studies the only areas to perform above architectural distinction. the less than 20 per cent band—indicating Contrasts are apparent in terms of the indi- that in the UK and Ireland households are vidual indicator performances. The analysis falling well short of the 40 per cent recycling demonstrates that the three waterfront case target. As expected, the three cultural case studies demonstrate a high ratio of open studies substantially outperformed the water- space to built form, whilst the cultural case front areas regarding the conservation of studies balance this with having a much built heritage, but conversely were lower higher ratio of converted/redeveloped build- ranked in terms of incorporating environ- ings to new build. Nevertheless, all case mental design into the building stock. With studies illustrate high density, a balance of the exception of Ciutat Vella and Temple uses (apart from Dublin Docklands) and the Bar, the case studies illustrated that little or complete removal of any contaminated land. no reclamation of building materials oc- The office rental indicator, with the excep- curred. This finding is somewhat surprising tion of Temple Bar, implies that favourable in sustainability terms, given the high levels rents are achievable with relatively high lev- of land filling of such waste. els of occupancy in areas that previously were classified as weak market locations. Buildings and land use. The best overall The performance of the property variables performance in five of the case studies is for reflects the acceptable levels of design and the Buildings and land use indicator group, quality of the final built product. However, reflecting the emphasis of these particular the main vulnerability from a sustainability  MEASURING URBAN REGENERATION, PART 2 765 Table 6. Buildings and land use scoring performance Buildings and land use indicators LS CQ DD TB OV CV Ratio of open space: built form 10 6 10 4 10 6 Ratio of converted buildings: new build 2 10 4 10 2 6 Reclamation of contaminated land 10 10 10 10 10 10 Density levels in relation to plot size 8 10 10 8 8 10 Balance of uses 8 6 4 10 8 10 Occupancy levels 8 6 10 8 6 10 Office rental v. CBD rents 8 8 8 2 10 8 Quality of final product 8 7 8 6 8 7 Design quality 8 6 7 9 9 7 Quality of public space 6 6 5 4 8 5 Usage of public space 6 8 6 6 8 6 Quality of private space 6 4 5 3 7 4 Total score 88 87 87 80 94 89 Percentage score 73 73 73 67 78 74 Weighted score (weighting factor 18.9) 1663.2 1644.3 1644.3 1512.0 1776.6 1682.1 Ranking 3 4 4 6 1 2 perspective, shown by the rankings, is the port links and travelling habits in relation to perception of poorer quality private space in leisure activities perform well. However, four of the case studies, the exceptions being with the exceptions of Temple Bar, the the Olympic Village and Laganside. Olympic Village and to lesser degree in La- ganside, the land allocation dedicated Transport and mobility. Transport and mo- specifically for pedestrian space and move- bility indicators encapsulate matters pertain- ment is ranked low with the converse that too ing to infrastructure improvement, travel high a proportion of land is devoted to roads. habits in relation to a range of activities and For residential car parking provision, the car parking provision. This indicator set pro- scores reflect that fewer spaces are allocated duced a varied performance across the case within schemes, whereas commercial car studies, with only the Olympic Village (73 parking provision points towards an overpro- per cent) and Temple Bar (70 per cent) pro- vision with the result that the public transport ducing high scores reflecting their excellent option becomes less lucrative. This raises internal accessibility and proximity advan- particular concern in meeting sustainability tages within their respective city centres objectives. Clearly, further deliverability is (Table 7). The Dublin Docklands was ranked required across the case studies in integrated third (63 per cent) ahead of Cathedral Quar- land-use and transport planning. ter, Belfast (62 per cent). Laganside (56 per cent) had the lowest score for Transport and Community benefits. Community benefits mobility, closely followed by Ciutat Vella capture a highly variable range of indicators, (58 per cent), making the Belfast case studies with the performance of Temple Bar (78 per the weakest collective performers. Given that cent) significantly ahead of the next-highest- a gap of 17 per cent separates the top and ranked case studies—namely, Ciutat Vella bottom performers, there is ample evidence (67 per cent) and Olympic Village (66 per to suggest that the importance placed upon cent). The high score associated with Temple accessibility and discouraging private car use Bar reflects attempts to foster close links differs greatly between case-study locations. with the community (Table 8). In compari- Those indicators measuring public trans- son, the Dublin Docklands (54 per cent) had 766 LESLEY HEMPHILL ET AL. Table 7. Transport and mobility scoring performance Transport and mobility indicators LS CQ DD TB OV CV Land devoted to roads 4 6 8 4 6 4 Land devoted to pedestrians 6 2 2 8 8 2 Road improvements 8 5 4 5 7 5 Work travelling habits 4 4 6 6 6 6 Leisure travelling habits 6 10 10 10 10 10 Public transport links 8 8 10 7 10 8 Car-parking provision—residential 6 8 8 8 8 6 Car-parking provision—commercial 4 8 4 8 4 4 Integration of land use and public transport 5 5 5 7 7 7 Total score 51 56 57 63 66 52 Percentage score 56 62 63 70 73 58 Weighted score (weighting factor 22.1) 1127.1 1237.6 1259.7 1392.3 1458.6 1149.2 Ranking 6 4 3 2 1 5 Table 8. Community benefits scoring performance Community benefits indicators LS CQ DD TB OV CV Access to open space 8 8 8 10 10 10 Access to leisure facilities 8 6 8 10 10 10 Access to retail facilities 8 10 8 10 4 6 Access to educational needs 4 8 4 8 6 8 Access to medical facilities 2 6 4 4 10 6 Access to entertainment facilities 8 8 6 10 8 6 Access to cultural facilities 4 4 4 10 4 6 Access to housing 6 4 6 6 7 7 On-site retail facilities 4 5 5 8 5 8 LA21 effectiveness 7 4 6 6 8 6 Community ownership 4 5 5 6 5 6 Community group involvement 3 1 1 5 2 1 Total score 66 69 65 93 79 80 Percentage score 55 58 54 78 66 67 Weighted score (weighting factor 20.00) 1320.0 1380.0 1300.0 1860.0 1580.0 1600.0 Ranking 5 4 6 1 3 2 the lowest score followed by the two Belfast On an individual indicator basis, there is a examples of Laganside (55 per cent) and low level of community group involvement Cathedral Quarter (58 per cent). For Dublin and, at best, only partly developed com- Docklands and Laganside, this in part reflects munity ownership of the schemes across all the predominant emphasis upon a physical case studies reflecting that, with the excep- development programme particularly in the tion of the Ciutat Vella, none is a truly early stages of the regeneration process. The established residential community area. In scores indicate that more attention needs to terms of accessibility, the outcome is vari- be placed upon improving community rela- able. Certain facilities such as retailing, tions at the outset of regeneration schemes leisure and entertainment perform at reason- rather than being slowly addressed in later ably encouraging levels in terms of sustain- phases. ability principles, although more alarmingly  MEASURING URBAN REGENERATION, PART 2 767 Table 9. Overall points scoring summary Case-study location Weighted score Percentage score Scaling Overall ranking Laganside 6460.8 62 ‘Good’ (5) Cathedral Quarter 6387.9 62 ‘Good’ (6) Dublin Docklands 6494.0 63 ‘Good’ (4) Temple Bar 6741.3 65 ‘Good’ (3) Olympic Village 7249.2 70 ‘Good–excellent’ (1) Ciutat Vella 6804.8 66 ‘Good’ (2) others such as education and medical facili- 5191.0 to 6125.38) indicating above-average ties produced a less satisfactory result apart performance, 60–69 per cent (6229.2 to from Olympic Village. Likewise, the fact 7163.58) classified as good, 70–79 per cent that accommodation in these areas has be- (7267.4 to 8201.78) good–excellent and come sought after and hence is highly priced above 80 per cent (weighted score 8305.6) can exclude lower-income households. This excellent. However, the spread of scores and is apparent in the variable performance of the associated banding have sufficient distinction housing accessibility indicator and raises so- also to enable application of the model to cial equity issues concerning—for example, areas considered to be failing from a regener- housing affordability. This grouping of indi- ation perspective. cators shows that out-turn performance is Each of the six case studies received high skewed more towards the remunerative de- total scores and can be classified as falling liverables, whereas the community compo- within the ‘good’ category, with the Olympic nents lag considerably with the result that Village Barcelona achieving the distinction sustainability objectives are not being fully of falling within the ‘good–excellent’ cate- embraced. gory. A relatively small percentage (8 per cent) separates the top and bottom perform- ers, reflecting the exemplar status of these Benchmarking Analysis three cities in terms of regeneration. On the The overall weighted scores, the summation basis of this analysis, the top-ranked case- of the product of the total points times the study location is the Olympic Village with 70 weighting factor, permit an analysis of each per cent (weighted score 7249.2) and Ciutat case study against the theoretical maximum Vella has the next-best performance with 66 possible score of 10 382. The median score per cent (weighted score 6804.8). Temple of 5191 is taken to represent average per- Bar ranks third with 65 per cent (weighted formance with the weighted points band score 6741.3). Outside the top three case (4152.8 to 5078.18) in the decile below the studies in terms of sustainability perform- median considered to be of below-average ance, the difference between the scores is performance. A case study with a weighted small. Dublin Docklands (63 per cent, total below 40 per cent of the maximum weighted score 6494) just edges out Lagan- possible score is rated as poor in relation to side (62 per cent, weighted score 6460.8) for sustainability practice. However, as all six fourth place, whilst the Cathedral Quarter (62 case studies were selected on the basis of per cent, weighted score 6387.9) although in recognised or potential achievements in re- the bottom position is still classified in sus- generation, it is important to have a better tainability terms as good (Table 9). It is the discrimination of weighted scores above the weighted score that an area or project re- median. Again, deciles permit a rational cate- ceives that is important. gorisation of scores into nominal groups with The findings illustrate that the model is the band 50–59 per cent (weighted score capable of discriminating between compara- 768 LESLEY HEMPHILL ET AL. tive overall performance levels on a case- data collection phase, substituting the orig- study basis. Significantly, each of the case inal score with the lower score where any studies confirmed that sustainability is be- difference occurred, with the converse occur- coming a more integral part of regeneration ring in the case of the higher variation. Both practice, with none of the areas below the sets of scores provide an alternative view of ‘good’ category. Furthermore, no noticeable performance and represent the extreme of the pattern was established between those case distribution of scores. studies undertaken in waterfront locations The results of the sensitivity analysis illus- compared with the more culturally based trate the effect that a change in the points projects; rather, differences are more related allocated to any indicator would have on the to the city than defined locations within a weighted scores and relative ranking of the city. case studies. In terms of the low variation, The case studies that have been completed some case-study areas proved to be more or those nearing completion provided the sensitive to a change in points total than best performances, notably the Olympic Vil- others (Table 10). For example, the Olympic lage (70 per cent), Ciutat Vella (66 per cent) Village shows the smallest percentage drop and Temple Bar (65 per cent), as these areas (4 per cent) in the overall performance level benefit from more established markets and indicating that it is the least sensitive to a fall greater investor, resident and employee in points total. Further confirmation of top confidence. This suggests that the perform- ranking for Olympic Village (66 per cent) is ance levels of the other areas should improve illustrated by an increase in the gap (to 6 per as schemes mature. In this respect, Dublin cent) to the next-highest-scored location, Docklands (63 per cent), Laganside (62 per Ciutat Vella (60 per cent), on the basis of cent) and Cathedral Quarter (62 per cent) lowest score variation. The largest drop in have performed well given these constraints points was experienced by Laganside and and just lag the performance levels set by the Cathedral Quarter (8 per cent), followed by top-ranked case studies. Temple Bar (7 per cent), resulting in both Belfast case studies displaying a similar per- formance level (54 per cent of possible Sensitivity Analysis weighted score) and confirming their relative The regeneration characteristics evaluated by status as the weakest performers, although the indicators are complex, with a degree of still above average. The utilisation of the low qualitative evidence from specific expert score variation also shows a narrowing of the groups as discussed in the accompanying gap between the Dublin case studies to 1 per paper. Hence, sensitivity analysis is deployed cent, demonstrating that the Temple Bar re- to allow for any uncertainty and to analyse sults were more sensitive to change than the the effect that a change in the scores may Docklands (Table 10). have on the overall result. Sensitivity simula- Perhaps the most significant result of ap- tions are undertaken whereby higher and plying the low score variation is the effect on lower variations of the final scores of several the overall ranking. In all cases, the appli- individual indicators were chosen to test cation of the low score variation saw a real- what impact, if any, this would exert on the location of the case studies from the ‘good’ overall final points scoring. This demon- category to the lower ‘above average’ tier of strates how sensitive the results are to a the benchmark scale and the top-ranked change in the individual indicator scores and Olympic Village area (6823.9) was reallo- indicates whether or not a particular case cated to the upper half of the ‘good’ cate- study should be reallocated to another per- gory. Of the case studies reallocated to the formance level on the benchmark scale. The ‘above average’ category, Ciutat Vella lower score variation was taken from alterna- (6195.7), Temple Bar (5982.8) and the tive pessimistic responses received in the Dublin Docklands (5948.3) are in the upper  MEASURING URBAN REGENERATION, PART 2 769 Table 10. Comparison and ranking of sensitivity analysis (low variation) Case-study location Weighted Percentage Percentage Scaling Original Low score score difference category ranking ranking Laganside 5588.5 54 -8 ‘Average–good’ (5) (5) Cathedral Quarter 5485.5 54 -8 ‘Average–good’ (6) (6) Dublin Docklands 5948.3 57 -6 ‘Average–good’ (4) (4) Temple Bar 5982.8 58 -7 ‘Average–good’ (3) (3) Olympic Village 6823.9 66 -4 ‘Good’ (1) (1) Ciutat Vella 6195.7 60 -6 ‘Average–good’ (2) (2) Table 11. Comparison and ranking of sensitivity analysis (high variation) Case study location Weighted Percentage Percentage Scaling category Original High score score difference ranking ranking Laganside 7003.9 68 ⫹6 ‘Good’ (5) (5) Cathedral Quarter 7164.7 69 ⫹7 ‘Good’ (6) (3) Dublin Docklands 6788.0 65 ⫹2 ‘Good’ (4) (6) Temple Bar 7115.8 69 ⫹4 ‘Good’ (3) (4) Olympic Village 7849.9 76 ⫹6 ‘Good–excellent’ (1) (1) Ciutat Vella 7251.3 70 ⫹4 ‘Good–excellent’ (2) (2) half, whilst Laganside (5588.5) and Ca- of the ‘good–excellent’ tier with the other thedral Quarter (5485.5) are placed in the case studies consolidating their position middle to lower part (Table 10). The analysis within the respective categories, notably the suggests that the overall scores awarded for Olympic Village (7849.9), Cathedral Quarter the case studies are sensitive to a downward (7164.7), Temple Bar (7115.8) and Lagan- change in points total, with all six experienc- side (7003.9). The Dublin Docklands experi- ing a reallocation to the next lower category. enced little change in weighted points total, Concerning the high score variation, some with only a slight improvement to the mid case studies demonstrated a greater sensi- position of the ‘good’ category (Table 11). tivity to points change than others (Table The overall scores awarded for the case stud- 11). The Dublin Docklands showed the ies have proved to be less sensitive to an smallest increase (2 per cent) in points total, upward change in points total, suggesting followed by Temple Bar and Ciutat Vella (4 that respondents may have adopted optimis- per cent), illustrating that the Docklands is tic assessments of regeneration areas in their the least sensitive to a rise in points total original response to particular indicators. from the high score variation. Cathedral As none of the case studies experienced Quarter (7 per cent), the Olympic Village greater than a 7 per cent change in the points and Laganside (6 per cent) experience the total allocated by selecting either the high or greatest increase in points. The analysis low score variations, it can be concluded that based on a high score variation establishes the original scores awarded can be treated Cathedral Quarter (69 per cent), demonstrat- with a high degree of confidence. Further- ing its latent potential, as the third-ranked more, the overall rankings developed from case-study area behind the Olympic Village the sensitivity analysis showed very little (76 per cent) and Ciutat Vella (70 per cent), movement, with the exception of the Ca- which consolidate their positions at the top. thedral Quarter case, which experienced a The application of the high score variation major upward change attributable to its in- moved Ciutat Vella (7251.3) to the margins herent potential. The outcome offers con- 770 LESLEY HEMPHILL ET AL. siderable support to the scores allocated and reliance to be placed upon the weights as- the rankings developed as well as the re- cribed to the components tier. In addition, the liability of the data sources used. model has flexibility in that parameters can be changed to reflect new standards arising from shifts in policy or technological devel- Conclusions opments, thereby contributing to its effec- Within urban regeneration policy and prac- tiveness in the evaluation of mechanisms in tice, the application of performance indica- creating sustainable urban environments. tors is becoming increasingly significant in The weightings applied to the respective measuring sustainability outputs. However, components enhance the utility of the model the use of methodologies to benchmark and as a comparative tool in the analysis of sus- assess the performance of sustainability re- tainability practice and outcomes. In this re- mains largely underresearched. Weighted in- spect, the model is capable of application dicators and a points scoring framework across regeneration areas and projects of dif- developed in the preceding paper are applied fering size, scale, level of perceived success/ to both waterfront-led and culture-led regen- delivery and at different phases of a eration schemes in three European cities to particular initiative. Indeed, the model is use- measure achievements in the delivery of sus- able as a tool to test at different stages the tainable developments/environments. The degree to which a particular regeneration study demonstrates that the more established project is adhering to the principles of sus- and mature regeneration locations (Olympic tainability. Thus, at strategic points over the Village, Ciutat Vella and Temple Bar) de- life cycle of the project, the model could be liver better sustainability performance than utilised to inform and influence strategy in a those areas which are less well developed, corrective capacity by highlighting any highlighting the importance of timing in re- significant divergence from sustainability ob- generation areas. The use of sensitivity jectives. Ideally, a project/regeneration area analysis to test the various options, particu- should be surveyed before mid term to en- larly for those indicators relying on qualita- sure that sufficient time is available to im- tive evidence, indicates that the points score plement change if scores are low implying allocated to the various case studies in the that the scheme that is unlikely to achieve empirical investigation can be treated with sustainability targets. In theory, resurvey confidence. Significantly, the analysis shows could potentially occur at several stages over that both of the Barcelona case studies, the the period of a project as well as forming part Olympic Village and Ciutat Vella, and Tem- of the overall evaluation at the end of the ple Bar, Dublin, display a high degree of initiative. adherence to sustainability principles. Potential policy benefits stem from the A key parameter of the hierarchical model model’s holistic approach and its applicabil- developed in the accompanying paper and ity across different regeneration mechanisms. tested in this analysis is its inclusiveness in In assessing the extent to which regeneration terms of 5 indicator sets in the components areas are conforming to sustainability cri- tier and 52 separate indicators. The robust- teria, the model provides a valuable tool in ness of the model is demonstrated by its policy analysis. Application, as demonstrated application in different regeneration typolo- by the case studies, illustrates where good gies (waterfronts and cultural quarters) and practice is occurring—notably, in the in different urban contexts in terms of city Barcelona examples and particularly the characteristics, institutional structures and Olympic Village. An added feature and po- policy frameworks. Development of the tentially of greater significance is the ability model through the use of expert opinion is a of the model to operate at a components level key aspect of the methodological approach, which, from a policy perspective, discrimi- the comprehensive nature of which allows nates between those aspects where the regen-  MEASURING URBAN REGENERATION, PART 2 771 eration initiative is working well and those sure success, Urban Environment Today, 62(21 matters which require further policy inter- January), p. 13. BROWNHILL, D. and RAO, S. (2002) A Sustainabil- vention. From the analysis contained in this ity Checklist for Developments: A Common paper, the strong performance of Laganside Framework for Developers and Local Authori- (Belfast) in terms of Economy and work is ties. Watford: CRC Ltd. highlighted, but equally apparent is the BURNS, R. B. (2000) Introduction to Research weaker performance in relation to Com- Methods. London: Sage Publications Ltd. CARLEY, M. and KIRK, K. (1998) Sustainable by munity benefits and Transport and mobility. 2020? A Strategic Approach to Urban Regen- Hence, policy priorities in Laganside need to eration for Britain’s Cities. Bristol: The Policy be redirected to the latter issues, whereas in Press, for the Joseph Rowntree Foundation. the case of Temple Bar (Dublin) the model DDDA (DUBLIN DOCKLANDS DEVELOPMENT AUTH- ORITY) (1997) Dublin Docklands Area—Master highlights a converse situation. Plan 1997. Dublin: DDDA. The capacity of the model to drill down DETR (DEPARTMENT OF THE ENVIRONMENT, and examine components is important from a TRANSPORT AND THE REGIONS) (1998a) Sustain- policy and decision-making perspective, par- able Regeneration Good Practice Guide. Lon- ticularly in the allocation and targeting of don: HMSO. funds and other resources by allowing pol- DETR (1998b) Planning for the Communities of the Future. London: HMSO. icy-makers to identify areas requiring action. DETR (1998c) Planning for Sustainable Develop- This in turn offers the possibility to influence ment: Towards Better Practice. London: future performance and guide policy and HMSO. practice. On a wider perspective, the appli- DETR (1998d) Sustainability Counts. London: cability of performance indicators allows HMSO. DETR (1999a) Sustainable Development Strategy policy approaches to be assessed on their for the UK: A Better Quality of Life. London: ability to deliver sustainable development HMSO. and assess whether partnership structures are DETR (1999b) Towards an Urban Renaissance: effective. Regeneration performance outputs Final Report of the Urban Task Force. London: are increasingly expected to show joined-up DETR. DETR (2000) Our Towns and Cities: The Future holistic thinking across programmes by pro- Delivering an Urban Renaissance. London: moting physical, social, economic and en- HMSO. vironmental criteria. The indicator-based DOE (DEPARTMENT OF THE ENVIRONMENT) (1994) model advocated and developed in these two Sustainable Development: The UK Strategy. papers allows an evaluation of the extent to Cm 2426. London: HMSO. EUROPEAN COMMISSION DGXI (1996) European which integration is being achieved and sus- Sustainable Cities. Expert Group on the Urban tainability objectives delivered within urban Environment. Luxembourg: European Com- regeneration. mission. GAHAN, J. (1993) Fiscal measures in promoting urban regeneration: Custom House Docks, Dublin, in: J. N. BERRY, W. S. MCGREAL and References W. DEDDIS (Eds) Urban Regeneration, Prop- erty Investment and Development, pp. 95–107. AUDIT COMMISSION (2000) Getting Better All the London: E&FN Spon. Time: Making Benchmarking Work. London: HAKIM, C. (2000) Research Design: Successful HMSO. Designs for Social and Economic Research. BERRY, J. N. and MCGREAL, W. S. (1992) Fiscal London: Routledge. mechanisms and inner city development, Jour- IMRIE, R. and THOMAS, H. (1995) Changes in nal of Property Finance, 3(4), pp. 543–552. Local Governance and their Implications for BERRY, J. N. and MCGREAL, W. S. (1993) Public Urban Policy Evaluation, in: R. HAMBLETON sector initiatives in the regeneration of Belfast, and H. THOMAS (Eds) Urban Policy Evalu- in: J. N. Berry, W. S. MCGREAL and W. DEDDIS ation: Challenge and Change, pp. 123–138. (Eds) Urban Regeneration, Property Invest- London: Paul Chapman Publishing. ment and Development, pp. 193–214. London: MONTGOMERY, J. (1995) The story of Temple Bar: E&FN Spon. creating Dublin’s cultural quarter, Planning BRISTOW, H. (1999) Developing the tools to mea- Practice and Research, 10(2) pp. 135–172. 772 LESLEY HEMPHILL ET AL. ODPM (OFFICE OF THE DEPUTY PRIME MINISTER) TUROK, I. (1989) Evaluation and understanding in (2002) Best Value Performance Indicators local economic policy, Urban Studies, 26, 2002/2003. London: ODPM. pp. 587–606. OECD (ORGANISATION FOR ECONOMIC CO-OPER- WONG, C. (2002) Developing indicators to inform ATION AND DEVELOPMENT) (2000) Urban Re- local economic development in England, Ur- naissance: Belfast’s Lessons for Policy and ban Studies, 39(10), pp. 1833–1863. Partnership. Paris: OECD. YIN, R. K. (1984) Case Study Research: Design SUDMAN, S. (1976) Applied Sampling. New York: and Methods. London: Sage. Academic Press.