Embedding Building Performance Evaluation in a Medium-sized Architectural Practice: A Soft Landings Approach

1.     INTRODUCTION

Research has shown that there is a substantial gap between the design intentions and measured performance of new buildings in the UK, with some sectors producing more than twice their predicted carbon emissions [1] [2]. This gap could preclude achieving the carbon reduction milestones and timelines set forth by public policy [3], as buildings’ operational energy demands account for nearly half of carbon emissions in the UK [4], and simulated performance predictions are relied upon to ensure regulatory compliance.

Building performance evaluation (BPE) has previously been defined as “the act of evaluating buildings in a systemic and rigorous manner after they have been built and occupied for some time” [5]. This current performance gap between design and reality defines a clear need for in-use feedback of building performance on shorter timelines than previous BPE methods have pursued to ensure the design, construction and project delivery methods employed are capable of meeting mandatory targets.

In practice, BPE sits between quality assurance, building management, customer service and research and development, depending on the methods employed. The lack of clarity in who benefits makes it unclear who on a project team is responsible for the cost. Several researchers have previously acknowledged the difficulty in pursuing BPE as a professional service [6], [7]. Another impediment is the lack of mandatory requirements for BPE. The process is not requisite, either through regulation or professional bodies, despite research showing both a clear need for industry-wide feedback, and a client expectation of significant aftercare, extending beyond normal contractual timelines, as a standard service from designers [8].

Until recently, BPE has mostly been an after-thought, rather than an integral part of the project delivery. Whilst BPE studies will result in recommendations for improvements, there is often little or no money or time set aside for addressing these by the design and construction team.

This paper describes the initial development, deployment and feedback relating to a bespoke set of Soft Landings tools [9] as part of a two year Knowledge Transfer Partnership between a medium-size architectural practice in the UK, Architype Ltd, and Oxford Brookes University, which aims to address these issues.

2.     STUDY APPROACH

2.1.   Structure

The study uniquely applies standard methods of post-occupancy investigation into the project delivery process to act as effective means to quality control the built product and aid in building handover.

Case study sites were selected for being typical for the architectural practice and studied across multiple stages of design, construction and occupation. This allowed pilot testing for integrating traditional post-occupancy evaluation methodologies and tools (Table 1) into the project delivery process to help transform them into diagnostic and quality assurance methods used during construction. This also identified the level of contractual, client, fee and contractor support required to integrate the procedures into future projects.

Table 1:     Investigative methods, versus Diagnostic and Quality Assurance Feedback sought using an integrated BPE process.

Quality Feedback
Methods	Operational		Construction		Design
ENERGY
TM-22 [11]		X		X	X
½ Hourly Demand Profiles		X		X	X
Heating Degree Day (HDD) Analysis [7]		X			X
Discrete Studies				X	X
INTERNAL ENVIRONMENT
Air Temp & Relative Humidity (RH) Monitoring		X			X
Carbon Dioxide (CO2)& Ventilation Rate		X		X	X
Occupant Survey		X			X
Daylight Study					X
ENVELOPE QUALITY AND ASSEMBLY
Thermal Imaging				X	X
Air Pressure Test				X	X
USE & USABILITY
Occupant Survey		X			X
Semi-Structured Interview		X		X	X
Photographic Walk Through		X			X
Controls Review		X			X
Residency		X		X	X

2.2.   Case Studies

Seven key sites (a housing complex, a refurbished office building, three primary schools and two children’s centres) received extensive investigation. Four secondary sites (two primary schools, one children’s centre and one office) received only one or two methods of investigation due to their particular phase of completion. All buildings are naturally ventilated, with some minor cooling in server spaces. The sites vary in terms of environmental aspirations and passive and active design strategies.

2.3.   Methods & Diagnostics

The study pursued minimally invasive investigation methods to reduce cost, risk, and disturbance to the end users. The methods evaluated energy use, internal environment, fabric quality and assembly, occupant responses, use and usability. The methods employed were similar for all sites, with the exception of the housing. As the research is still in progress, not all methods have yet been used on all sites.

Energy assessments utilised weekly meter logs using an online logging and analysis platform [10], which employs a standard method of heating degree-day (HDD) analysis [11], using an advanced degree day dataset for UK weather stations [12]. Half-hourly energy demand profiles were assessed, where available, for month-long occupied periods and two-week unoccupied periods in summer and winter to check for aberrant demand behaviour [13]. CIBSE TM-22’s [14] will be conducted this winter, using the weekly meter logs and a standard energy walkthrough checklist [15]. Discrete quantitative studies were conducted where necessary to investigate abnormal energy behaviour.

Environmental assessments included logging internal air temperature and relative humidity across key sections of the building, at half-hourly intervals, and using local met station weather data for comparison with external conditions. A CO₂ ventilation rate and in-use concentration study, based on Mumovic’s method [16], is aimed at giving operational feedback to aid in managing CO₂ levels. Lighting field studies are planned for winter 2011 to extract daylight profiles across key building sections.

Fabric quality was assessed on all sites with a thermal imaging survey, by imaging the interior and exterior building surfaces pre-dawn on mid-winter mornings, sometimes with the contractor present. The housing site will undergo additional testing in winter 2011, including a tracer gas concentration decay analysis and empirical U-value analysis by logging the heat flux through key assemblies.

A series of studies assessed usability. Photographic walk-through surveys documented usage that varied from design intentions, end-user controls, health and safety, ergonomic and accessibility issues. Occupant surveys included a cartoon-based thermal comfort, acoustic and lighting survey for pupils, after Woolard [17], to plot comfort profiles during a week in the first occupied summer and winter, and a bespoke 7-point control, comfort and use survey for adults after the completion of two complete, occupied heating seasons. Discussions and semi-structured interviews are ongoing with premises managers to assess the buildings’ manageability. A short, formal management and maintenance survey is under development for buildings beyond their defects period. The housing study will also employ a unique occupant behaviour analysis [18].

Additionally, a Soft Landings ‘residency’ [9] was trialled to assess issues surrounding handover, and initial occupation to develop methods for improving that portion of the building delivery process. This involved visiting one study building, half-day a week, for the initial 6 weeks following occupation, and informally visiting others, several times during the first year. The site visits involved photographic walk-throughs, semi-structured discussions with the site managers and school heads, and informal discussions with the teachers and other adults.

3.     Construction Phase

3.1.   Issues

A number of construction defects, which had not otherwise been apparent, were found via thermal imaging in several sites, and corrected where possible. In one Children’s Centre severe infiltration was found on a courtyard-facing wall. The live camera image was shown to the contractor and further investigation immediately agreed with the project architect. The contractor found two defects, including a lack of sealing around the doors and windows, and a hole through the wall where electrical wiring was run outside, and not sealed before finishing the wall (Fig. 1). The contractor addressed the issue within 2 weeks of the survey visit, demonstrating the impact of using thermal imagery during the construction phase.

Figure 1: Infiltration at the base of an exterior wall with wood clapboard interior finish.

Other issues identified included areas where insulation had been removed and poorly re-installed by services trades (addressing construction quality), as well as examples of thermal bridging due to detailing and choice of material and fenestration strategy (providing design feedback).

3.2.   Resolutions

The thermal imaging process was adapted into a standard specification that requires imaging the building during the first winter it has a completed thermal envelope, which has been included in contract documents on new projects.

Additionally, through interviews with the design teams it was identified that air tightness targets were not always achieved. While this research project did not have funding to field test air tightness, it supported an improved specification for multiple air pressure tests during the construction phase, to accommodate corrections during the construction process. Both specifications were incorporated into the tender documentation for a number of projects, and the contractors accepted the specifications without note.

4.     Pre-handover Phase

Pre-handover was not observed due to the time-intensity of the commissioning process; conclusions for improvement were inferred from the initial occupation period, by identifying where pre-emptive support could have mitigated issues that surfaced during the residency.

4.1.   Issues

A major issue noted was the lack of planning for the move-in and the induction of end users. While senior users went through training and induction with the contractor, as per the commissioning requirements, there was no plan in place to pass that knowledge down to non-technical end users. They found the new building immediately confusing and went for several days without simple things such as keys to open windows, or training in how to operate basic systems.

4.2.   Resolutions

This has triggered the assembly of guidelines for the building handover, which prompt building management in the pre-handover phase with issues they need to plan for to get the building up and running efficiently during and after the move. Larger commercial clients may use move management consultants, however these are not common for smaller occupancies like schools.

Also, a simple non-technical end-user guide will be assembled, over and above compulsory technical guidance, using pre-handover photos of building controls with a few simple pages of guidelines to explain key control and energy-saving features to end users.

5.     handover and initial occupation

5.1.   Issues

Several site visits revealed evidence that end users were unprepared for typical teething issues that would be expected from any new building, and how they can be resolved. As examples, users twice mistook settling cracks for indications that a wall was structurally unsound, and others thought that having to pull doors closed, as the acoustic door seals had not yet broken in, was a permanent defect.

Few sites had any self-initiated energy management strategies, and those that existed were more related to energy reporting than management and lacked any effective analysis or communication strategies that empowered end users to reduce consumption.

The residency period was not incorporated in the contract documents of the study building as it was under construction before this research project began. The lack of early phase preparation and contract coverage for the residency highlighted the need for full contractor support and contractual mechanisms that ensure “client-facing” defects are resolved quickly. When issues were not resolved quickly, users became sceptical of the resident’s effectiveness and value.

5.2.   Resolutions

A plain language “Welcome Letter” now explains common first-year teething issues, explains whom end users should contact, and what simple actions might be taken to resolve them. The intent is to manage users expectations, keep them informed of their environment, and aid them in taking ownership and control of the building.

Arranging student councils to take charge of the schools’ energy management has prevented the overburdening of site managers and deployed several dozen enthusiastic energy managers in each building. The researcher spent a full day with each of two student councils, teaching them about energy in buildings and conducting demonstrations with a plug-meter. A pair of pupils took charge of each sub-metering zone, developing strategies for reducing energy use in their zone, and designing energy-saving posters to mount around the school. The councils’ weekly responsibility is to input, monitor and reduce energy use in their zones, using the meter-logging website [10]. Three schools were very receptive to this method of energy management, one ignored the initiative, and one outright rejected it. To support its adoption on future projects, this method is now presented as an option in design consultation, to ensure energy management strategies are designed in concert with the metering equipment and sub-meting strategies.

A newsletter was circulated during the fourth week of the residency, listing the issues raised by users and explaining their current status. Users appreciated being kept informed even when issues were not resolved straight away, which is similar to other research findings [19]. In addition to gaining contractor support in early project stages, and contract conditions that specify response times for resolving certain types of defects, a bi-weekly newsletter will be part of the evolving method.

6.     First year of occupation

6.1.   Issues

As part of the assessments, a graphic comfort survey was conducted of the pupils in 2-3 classes in four schools. The format was a “raise of hands” 7-point thermal, visual and acoustic comfort survey, which were to be conducted twice daily for a week. The survey was successfully pilot-tested in each school before its distribution, however only one set out of 11 was returned complete. Teachers gave various reasons for not completing them such as swapping classrooms, teacher illness, school inspections, and in the case of younger children, the survey caused too much distraction and took too much time away from lessons. Due to the poor result, this has been dropped from the overall assessment strategy.

Several buildings were handed over with environmental control issues, as these are generally too minor to withhold Practical Completion. At handover in one school, a box of unlabelled remote controls intended to operate various roof vents was left with the site manager who could not match the remotes to the specific vents. This left the stack ventilation effectively disabled. In another building, occupants felt that clerestory windows, controlled by the building management system (BMS) was a nuisance and did not provide enough air. They disabled automated vents and opened manual windows and exterior doors to compensate.

There was a lack of simple building guidance to give controls technicians an overview of the building strategy before making adjustments. Seasonal commissioning was done without comprehensive historical performance data, and separate sub-contractors servicing different control systems, meaning adjustments were made myopically, with limited understanding of the effect that adjusting one system might have on another. Proprietary control technology, which log energy and environmental data, inhibit independent verifying and quality assuring control settings; meaning, controls sub-contractors effectively quality-assure themselves. Independent temperature logging and utility demand profiling revealed correctable, out of hours heating and electrical equipment operation.

These are all common findings in post-occupancy evaluations [20] but identifying them within the confines of an active contract should provide an opportunity to make corrections.

6.2.   Resolutions

With regard to commissioning ventilation devices, a one-day indoor air quality study assessed CO₂ concentrations, in-use ventilation rates and strategy capacity. This enabled corrective action on both schools, as a construction defect, and an operational change, respectably. The procedure goes well beyond standard seasonal commissioning and can effectively marry occupant expectations with system operations.

A suite of spot-check procedures have been developed to ensure that the basis for any corrective adjustments is a cohesive, complete picture of the building’s performance, to ensure the building is or can work towards operating within agreed tolerances. To date, the suite includes an independent TM-22 assessment [14] in advance of the 6-month seasonal commissioning session, temperature logging in key spaces, indoor air quality assessments, discussions with occupants, and the analysis of half-hourly utility demand profiles. Further resolutions and specific findings will be published as the project develops.

7.     year two and beyond

For buildings that have been occupied more than 2 years, a leaner study is recommended, employing only meter logging, which is now set up for all new buildings, a management and maintenance survey, an occupant survey, a photographic walk-through, and discrete studies, where particular assemblies or strategies are of interest.

As part of the study, however, older buildings underwent more intense studies to gather a broader range of feedback and understand the advantages and disadvantages of revisiting buildings after a standard handover process and several years of use.

7.1.   Issues

Discussions with occupants in one revisited site revealed lingering dissatisfaction with issues relating to the initial build and handover, which had occurred more than three years earlier for which various team members could have held or shared responsibility. Occupants’ entrenched opinions about the building made them less receptive to recommendations to improve performance, as they felt issues had been left unresolved too long, creating difficulty in cooperatively implementing solutions.

Concerns emerged that building managers may have perceived an architect-appointed post-occupancy researcher as an ‘admission of guilt’ for aspects of the building they felt were unsatisfactory, which triggered a duty to notify insurers of a potential claim. While this particular situation did not result in an insurance claim, it highlighted that any increase in notifications to insurers will result in higher premiums to cover the additional risk to the provider, putting individual firms within the design team at financial risk from BPE studies aimed at improving the building performance, even when no claim results.

7.2.   Resolutions

Rectifiable quality issues must be addressed before the design and construction team has disbanded. In the short term, this means integrating quality control specifications into contracts to ensure the level of performance that was specified is achievable with the building that is handed over, to seek concessions where necessary and ensure that shortfalls in performance are understood by the entire team.

8.     PROCESS improvementS

The process has pointed to several infrastructural changes to project delivery. These would allow BPE activities to be taken up as standard practice in a way that manages risk and ensures that quality issues are addressed by the whole team.

8.1.   Integrated Teams

The project is investigating integrated or “partnering” teams [26] to control the risk, cost and team support issues with integrating BPE and long-term quality assurance in project delivery. Integrated teams focus efforts on delivering a fit-for-purpose product. The pain/gain share scheme takes account of performance issues to ensure that quality assurance is conducted by pre-agreed means and methods and measured against project performance targets. This way, no one member of the design team is taking an independent or disproportionate risk for evaluation activities.

8.2.   Staged Handover

The standard retention during the defects period for UK contracts is 1.5 – 2.5% of contract value [21], [22]. This relatively modest sum does not necessarily encourage prompt resolution of defect, controls issues, or providing support to help the building become self-sufficient, placing a heavy reliance on the contractors approach to customer service to ensure a smooth handover.

Adding a Technical & Verified Handover stage, beyond the currently defined stage, with increased retentions for both, could provide financial justification and defined timelines for conducting quality checks and operational adjustments. This would require a fundamental change in standard building contracts in the UK.

8.3.   Policy guidance and Dissemination

Current UK building regulations require declaring specific energy performance targets, without accounting for unregulated loads, using accredited software with proven inconsistencies [23]. In order for the industry to comfortably adopt a process of integrated quality assurance and joint responsibility for performance, targets would need to include tolerances based on in-use data compared with simulated predictions. Defining realistic performance tolerances relies on matching predictive simulation with both short and long term performance evaluation. The lack of requisite BPE activities in the UK building regulations and its position as a voluntary requirement within professional bodies such as CIBSE and RIBA, inhibit affecting positive change in the quality of the built environment.

9.     Internal dissemination

The project’s success relies on setting up suitable internal dissemination processes that embeds the lessons learned back into the practice effectively, and number of these have already been developed.

9.1.   Knowledge sharing website

Lessons learned from BPE are now uploaded to an internal knowledge-sharing website. Constructed from the MediaWiki framework [24], the locally deployed, open source platform is highly flexible; third-party plug-ins can extend functionality, an existing global network can offer technical support, and the PHP-language allows bespoke development.

To share a lesson, the users complete a standardised form, similar to a customer profile on an e-commerce site. User permissions restrict editing abilities to prevent accidental deletion, and article “talk” pages provide a venue for casual discussion or disagreement. The current design aims to reduce maintenance by structuring the input logic to self-organize data, but some regular maintenance is required on master pages.

9.2.   Continuing professional development

The project initiated a series of internal, single-issue lectures using examples from the BPE studies to illustrate the problems. Single-issue lectures facilitate in-depth discussions and problem solving by illustrating how the same issue occurred in different circumstances, facilitating the analysis of contributing factors, and using redundancy to emphasise the importance of taking corrective action.

9.3.   ISO9001

As BPE and Soft Landings are both methods of quality controlling the built environment, these efforts are now part of Architype’s ISO9001 Quality Management System [25], which requires periodic reviews of quality issues. The reviews link to maintaining the knowledge-sharing site, ensuring that information in the database remains relevant to current practice.

10.  Conclusions

BPE and advanced handovers are essential to progress the UK construction industry’s quality output and ultimately reduce the carbon impact of the built environment. These efforts are achievable within current practice, but they require substantial team commitments to quality. Project teams need to agree succinct plans from early project stages to capture and apply lessons learned, ensure team support, plan for the time and cost of BPE activities, and manage risk. Improvements in contractual, regulatory and professional support for BPE could ensure quality improvements in British construction by reducing industry-wide risks, providing contractual mechanisms for assessment, and mandating certain BPE methods.

11.  ACKNOWLEDGEMENTS

The Technology Strategy Board and Architype fund this research. The authors acknowledge help from Bill Bordass, Adrian Leaman, Zack Gill, David Smart, and the Architects at Architype.

12.  REFERENCES

[1]    Pegg, I., 2007. Assessing the role of post-occupancy evaluation in the design environment – a case study approach. EngD. Brunel University.

[2]    CarbonBuzz an RIBA CIBSE Platform www.carbonbuzz.org, Acc’d: Nov. 8, 2010.

[3]    Climate Change Act 2008, (c.27), London:HMSO.

[4]    Department For Environment, Food And Rural Affairs, 2005. Securing the Future: Delivering UK Sustainable Development Strategy. London:HMSO.

[5]    Preiser, W, and Vischer, J. eds. 2005. Assessing Building Performance. Oxford:Elsevier.

[6]    Stevenson, F., 2009. Post-occupancy Evaluation and Sustainability: A Review, Proc. Institution of Civil Engineers Urban Design and Planning 162 Sept. Issue DP3 pp.123–130.

[7]    Bordass, B., and Leaman, A., 2004. PROBE: how it happened, what it found and did it get us anywhere? Proc. The Windsor Conference: Closing the Loop: Post Occupancy Evaluation – The Next Steps, Windsor.

[8]    The Usable Buildings Trust. Soft Landings for Schools Phase 1 Final Report on Cast Studies March 2010.

[9]    BSRIA, 2009. Soft Landings Framework

[10] Oxford University Environmental Change Institute, SMEasure beta www.smeasure.org.uk,

[11] The Carbon Trust, 2006. GPG310 v3 -Degree days for energy management — a practical introduction.

[12] Layberry, R., 2008. Degree days for building energy management-presentation of a new data set. Building Services Engineering Research & Technology, 29(å3), pp.273-282.

[13] Wright, A., Brown, N., 2008. The analysis and interpretation of half hourly utility data in UK buildings. Proc. Improving Energy Efficiency in Commercial Buildings Conf., Building Performance Congress, Frankfurt.

[14] CIBSE, 2006. Energy assessment and reporting method TM22:2006. CIBSE

[15] The Carbon Trust, 2004. GPG057 v3 – Conducting an Energy Walk-round of a School.

[16] Mumovic, D., Davies, M., Ridley, I., Altamirano-medina, H., Oreszczyn, T., 2009. A Methodology for Post-occupancy Evaluation of Ventilation Rates in Schools. Building Service Eng. Research and Tech. 30(2), 143-152.

[17] Woolard, D., 1981. The graphic scale of thermal sensation. Arch.Sc.Rev. 24(4), pp.90-93.

[18] Gill, Z., Tierney, M., Michael, J., Pegg I., and Allan N., 2010. Low-energy dwellings: the contribution of behaviours to actual performance, Building Research & Information, 38(5), pp.491-508.

[19] Leaman, A., 2001. What Occupants Want, Ch. 6 of Davis Langdon and Everest Institute of Directors Guide, Buildings that Work for Your Business

[20] The Probe Team. Probe Review Final Report 4. Strategic Conclusions. Building Use Studies, August 1999.

[21] The Joint Contracts Tribunal Ltd. 2009. Standard Building Contract With Quantities, Rev. 2, Thomson Reuters Ltd 4-20, pp.59.

[22] The Joint Contracts Tribunal Ltd. 2005. Minor Works Building Contract, Thomson Reuters Ltd. 4-5, pp.5

[23] Raslan, R., & Davies, M., 2009. Results Variability in Accredited Building Energy Performance Compliance Demonstration Software in the UK: an Inter-model Comparative Study. Journal of Building Performance Simulation. 3(3), pp 63-85.

[24] Wikimedia Foundation, www.mediawiki.org

[25] International Standards Office, 2008. ISO 9001 – Quality management systems – Requirements Electronic documents. Geneva: ISO.

[26] Department of Trade and Industry, Construction Task Force, 1998. Rethinking Construction. London:HMSO.