INTRODUCTION
NFP organizations set out to do well and to do it properly. But measuring the performance of an NFP organization can be a very difficult task. Trying to devise a measuring system can be a real challenge to someone coming from the for-profit sector where the indices are often just financial (though there are now some ratings systems which also cover sustainability and corporate social responsibility).
Here I will deal with the two main methods of measuring performance: assessing the performance in terms of the organization’s mission and execution, and then the more conventional (but also occasionally difficult) financial performance. These two forms help make sure that the organization is remaining true to itself. Measuring the performance of activities means that the organization is adhering to its overall vision.
Measuring the financial performance means making sure that it is financially sustainable. The experience in the for-profit sector is that most companies do not fail financially in one dramatic year: it is possible to detect the seeds of bankruptcy two or more years out from the final disaster. Similarly NFP organizations tend to slide into a genteel decline rather than suddenly fail in one year.
It concludes with a comment on the wider network of checks and balances in which organizations may also have their performance measured.
The bottom line is that being a director means stepping into a web of monitoring systems. It is a highly complex environment.
The essence of the problem is one of devising key performance indicators (KPIs). KPIs vary from one organization to another. A lot of things could be measured but how many measurements really assist directors to understand what is happening in the organization? Some KPIs are non-financial and some are financial. The challenge is to devise KPIs that are relevant and current, and measure what needs to be measured (rather than what can be measured, which may in the grand sweep of things not really tell the director much about the organization at all).
MISSION AND EXECUTION
Introduction
This section begins with an examination of the old ideas of measurement. It then has a case study of poverty in Australia to bring out some of the specific complexities involved in measurement calculations.
It then looks at a more systematic way of measuring performance and the need to develop new tools. Boards with the right directors can add much value here by bringing in skills from their other professions.
The section ends with a warning about “mission creep” and a reaffirmation of the importance of keeping focussed on the organization’s strategy.
The Old Era of Measurement
The essence of the measurement problem is that NFP organizations are trying to improve the conditions of humans or animals or the environment. Are we making a difference? People join NFP organizations because they do want to make a difference. Meanwhile there has also been in recent decades a financial incentive for improving the measurement of performance: governments and other funding bodies and donors want to know that their money is being well spent.
Traditionally the assessment was done virtually intuitively: providing a particular service for a particular client somehow felt the “right” thing to do. One problem with his approach was that intuition varied from one organization to another or perhaps from one staff member to another. There was no consistency.
Second, the response was often an immediate one: such as (in a welfare example) providing cash or food to deal with an immediate problem. There was little attempt to deal with the underlying problems, which could lead to complications[1]. In an animal example, cases of cruelty could be handled via immediate assistance to the animals – rather than asking why the animals were getting injured in the first place.
Third, the measurement was often just based on figures that could be collected easily: for example, number of meals issued, ratio of beds occupied per night expressed as a percentage, and number of clients visited per day. The figures were useful for showing trends of going up or down. But they gave little indication of the actual quality of the service or indeed what was the overall value of it.
Fourth, without an assessment of the cause of the problem, the care was done in repeatable episodes but without necessarily finding any way for the organization, so to speak, to work itself out of a job. Figures were provided but there was no narrative. There was no way of making sense of them or helping a board to ask broader strategic questions about the nature of the work and its effectiveness.
Fifth, for people coming out of the healthcare sector into the welfare sector there was a surprise in the poor quality of data collection.[2] The healthcare sector has made great progress in monitoring patients and methods of treatment and there are elaborate financial models (such as “casemix” and the creation of DRGs: diagnosis related groups). But in the welfare sector, the data were not consistent in their meta-data (definitions) and in the methods of collections over the years. This meant that comparison was difficult between organizations or within the same organization over a long-period of time. For example, there has long been disagreement over the number of homeless people in Australia.
Finally, most staff and volunteers in most organizations are just too overwhelmed by the immediate tasks at hand to stand back and ask the bigger questions relating to the service.
This is where a good board could add value. It takes us right to the heart of an organization’s strategy. A strategy-focused board would want to know the narrative behind the figures and what they meant. But most boards do not at present ask those questions; if nothing else it would seem all too complicated, as a study of the problem of poverty shows.
The Problem of Poverty
Australia is one of the wealthiest countries in the world and yet it still has poverty. Trying to understand this problem is a good case study of the problem of measurement.
First, there is the basic question of defining “poverty”. The United Nations has a generic definition for “absolute poverty” in the developing world. It was set at US$1 per day in 1990 and it was raised to US$1.25 in 2005.
The Whitlam Labor Government (1972-5) invited Professor Ronald Henderson to define “poverty” for the Australia. In December 1973 it was set at $62.70 per week. Henderson’s Melbourne Institute of Applied Economic and Social Research continue to update the Henderson poverty line.
But “poverty” is not just a financial matter. In a developed country like Australia it also has to include matters relating to education, health, access to services and infrastructure, social exclusion and access to social capital. For example, many Indigenous People living in Redfern, Sydney have some appalling economic and health indicators and yet they live within walking distance of some of Australia’s finest hospitals. Why won’t they make greater use of the facilities?
The second problem with measurement is the discussion over how people become poor in the first place (and so how an organization can help them devise their escape routes out of poverty). There is an extensive literature on this subject and no agreed consensus. Here is my approach.
There are basically three routes into poverty in Australia. First, there is an event which temporarily plunges a person into poverty. For example, a middle class woman suddenly loses her husband (who is the breadwinner) and becomes a single parent with young children. If she has enough resilience and social capital, she will eventually recover from the setback and return to a more comfortable lifestyle.
Second, there is a person who is born into poverty, lives in poverty, with no immediate family role models of people who have escaped from poverty. The risk is that this person will regard poverty as the norm, become reconciled to it and so, because of the welfare state, live to a ripe old age (hence the need nowadays for old age facilities for people who have lived their lives on the streets and have not perished at the young age that previous homeless people did). This person tends to cope with poverty better than the people in the first category who are taken by surprise by their plunge into poverty and may lack the coping skills required for surviving poverty. People in the second category have become reconciled to poverty and have no mental picture of their ever escaping it.
Finally, there are people who may well have come from a middle class background but who made some bad decisions (perhaps because of the wrong company they kept) which will colour their lives for many years to come. For example, a middle class person may have acquired a drug habit at university and then slipped into a life of crime and squalor. Early life decisions and accidents could shape much of the rest of their life.
These three broad types of clients require different approaches by organizations. There is no one system that will work with all three types of person. Here is where NFP organizations have the flexibility and ingenuity to deliver services to individuals and their families in a much better way than the standard government bureaucratic regimented system.
The New Era: Five Key Concepts
Over recent decades, then, there has been a growing sophistication in the assessment of the human needs and how best to respond to them. NFP organizations, government and donors are trying to devise more accurate ways of measuring performance.
There is not yet one set of agreed measurements for all organizations. Some NFP organizations continue simply to list in their annual report, say, the number of families assisted in one year, the number of children cared for by the organization that year, etc.
Ironically an indicator of “success” for a child welfare organization’s annual report might be that more children were cared for last year than in the previous year. The figure might be numerically accurate – but actually of little value. Legislation may have changed so that more children had to be taken into care, for example. All child welfare organizations would have reported more children in care and so this particular organization is no different in this benchmark than any of the others.
The number therefore tells us very little – but looks good when appealing to donors. In fact, an increase in workload could be a good sign (the organization is highly regarded by government and so more cases are being referred to it) or as a bad sign (the organization is working harder but it is not reducing its caseload). Whether it is a “good” or “bad” sign cannot be inferred from the figures alone. But no doubt the marketing department would argue that it is a “good” sign and so appeal for extra funds to cover the extra caseload.
Directors need to understand the underlying complexities of numbers and be able to drill down into their meaning. Here are five indices that help clarify an organization’s performance.
Ideally, a project should be measured along the lines of these five key concepts: objectives, input, outputs, outcomes and impact. They are brought together here in an example of assisting children with learning problems.
Objectives are the targets in the provision of services. In this example, it is the provision of reading support for children from financially disadvantaged backgrounds having learning problems at a school.
Inputs are the resources deployed within the organization: mainly number of volunteers and staff, and financial expenditure, deployed over a set amount of time. In this example, the project consisted of one staff member and 10 volunteers over a six-month duration at a cost $50,000.
Outputs are the immediate results of the inputs. In this example, it was the holding of 10 sets of weekly sessions of one hour each session over a six-month period with 20 children (two children per volunteer over the duration of the project), which came in on budget.
It is at this point that most annual reports finish. Actual and potential donors other funding bodies are left with the image of a hard-working organization assisting deserving children (which it could well be). But the story is not complete. It could go further.
Outcomes are the mid-term results, which are not necessarily seen immediately at the end of the activity. In this example (hopefully) there are educational improvements in the lives of these school children who are doing much better at school (which itself can be assessed via, for instance, school examination results, attendance rates, and class participation).
Impact is the most long-term aspect of a project. In this example (hopefully) some of the school children eventually escape their poverty and live in comfortable middle class prosperity and start to raise their own children with a new set of values and perspectives.
In this (very optimistic) example, everyone involved would regard the project as a great success.
Unfortunately, life is not that easy. First, the numerical assessment becomes more difficult as it moves from output to outcome to impact.
Second, there would be a growing complexity of cause and effect as the assessment reaches the impact stage. Did the children do well because of this particular project or did other factors also play a role. The technical terms here are endogenous (internal causes or origins) and exogenous (external causes or origins). Directors need to ask their CEO about these two sets of factors when assessing a programme’s effectiveness. It would be difficult to make an accurate assessment of the factors but it would put everyone on notice that there could be other factors at work.
Third, the full assessment is complicated because there is a need for a longitudinal study of all the children. This requires a high degree of contact with the clients over an extended period of time[3]. This is usually not possible: it is costly, time-consuming, and can be restricted by privacy legislation (which hinders the exchange of personal information on clients between organizations).
To conclude, most annual reports only tell a partial story: mainly of the effort made and the immediate output. Assessing outcomes and impacts are much more difficult but they are needed for the full story.
Developing the Right Tools
Board members from the for-profit sector, especially from companies dealing with multiple clients, may be able to offer some insights into how they maintain contacts with clients and their changing needs. It is an exciting time to become an NFP director because of the opportunity to participate in the development of the new assessment tools.
A director should question the CEO on how management is measuring the effectiveness of the organization’s programmes.
Bob Garratt warns about “moments of truth” in the for-profit sector.[4] This is when, for example, a customer asks for assistance from the maintenance department and a person in cynical voice asks the customer “Who sold you this then?” Customers will talk about negative “moments of truth” to their friends far more than their positive “moments of truth”. The challenge for directors is to find out how many moments of truth there are within the company. How does the director find out? Does the director even know who the customers are? Why do the customers buy from this company? “Customer-facing staff” – the staff who interact with the customers – are crucial for a company’s success. Constructive listening and systematic problem-solving for the customers have become essential elements in the learning systems of successful companies.
The challenge is to develop similar systems for the NFP sector. How does a director of an organization assisting, say, financially disadvantaged people know that the staff are making the most of the “moments of truth” when they engage with their clients? Does the organization have a research team that work with the “client-facing staff” and volunteers to try to learn from the interactions?[5]
To conclude, government, funding agencies and donors have increasingly high expectations of how measurement is conducted. This is an evolving field and board members should kept themselves alert to the changes.
Mission Creep
To complicate still more the discussion over the measurement of mission, there is the problem of “mission creep”. An organization starts out to do one thing and ends up with many other projects. For example, it may be attracted by the need to acquire new sources of funds. Here is an American example from Gregory Dees of Harvard Business School:
Like the proverbial tail wagging the dog, new sources of revenue can pull an organization away from its original social mission. Consider the YMCA. The association today generates substantial revenues by operating health and fitness facilities for middle-class families, but critics charge that the YMCA has lost sight of its mission to promote the spiritual, mental and social condition of young men”. [6]
But in fairness to that organization, if it is going to run welfare and educational services, then it may have to be ambitious in its financial activities. In the notorious words of the late Archbishop Paul Marcinkus “You can’t run a church on Hail Marys”.[7]
There is also the problem of when government and potential donors, aware of an organization’s high public standing, go to the organization with the offer of funding for a project just outside its normal remit. Many organizations would dispute that this is even a “problem” and would be happy just to receive the money.
A role for the board would be question whether a proposed project really does fit within the organization’s overall strategy. Some questions to stimulate that higher order thinking: “What is this organization trying to achieve?” “How will we know if we have been successful?” “How does this proposed project align with our existing strategy?” “If the proposed project takes us outside our normal remit, how will we be able to monitor it?”
The board should also ask about the exit strategy. For example, a change in government could lead to the abrupt end of that project’s funding, and the organization would be left to finance the project on its own. “Market conditions” in this context can often change faster than in the for-profit sector, where consumer sentiments are not as fickle as government policies. When a new government comes to power it may decide that the old government’s agreements should no longer apply and that the contracts will be allowed to lapse. It is risky being reliant upon government funding. The board should be constantly alert to potential changes in government funding and not be left in the lurch by a new government.
Debate within an NFP organization over whether a programme should be wound up is not just commercially driven. Such a decision will involve many non-financial factors and the discussions can become quite emotional. The debates are often more anguished than in the for-profit sector, where the financial results alone are usually the determining factor.
FINANCIAL PERFORMANCE
Introduction
Assessing financial performance is more than reading the figures produced for the board meeting. A director needs to understand the business, the subject matter in which the organization is involved, and to be able to interpret the financial reports in this wider context.
Financial competence is an acquired skill. It is vital that all directors have it. In the evaluation of directors, directors should take the opportunity to look for ways to request an upgrade to their skills. Accounting is a very complex subject and its rules are constantly changing. Directors need to be alert to the implications of the changes (rather than necessarily trying to become experts on the changes themselves) and to question the CEO if they do not understand something. For example, what is not covered by the directors and officers’ liability insurance cover.
This section will examine the financial performance task under four broad headings: maintaining proper books and records, financial reporting, monitoring the organization’s financial situation, and making sure that the organization does not trade while insolvent.
Maintaining Proper Books and Records
There is no one set regulation for how the books and records are to be kept because of the diversity of organizations and accounting systems. The requirements are that such books and records are to be kept and that directors are to have access to them. The records should be kept for seven years.
It also means that the appropriate financial reports are filed with appropriate government departments and agencies (and where necessary funding bodies) by the due dates.
These matters should all be covered by some form of procedures manual.
Directors should know the organization’s main sources of revenue. The organization may well have some limitations on donations (such as a refusal to accept donations from some types of businesses) and directors need to assured by the CEO that the marketing department other staff are aware of these limitations. Meanwhile, major donors need cultivating to encourage further donations, and so the staff should be doing this. Perhaps directors with good public standing may also be of assistance of liaising with the donors.
Similarly directors need to know the organization’s main costs. They will probably be on staff – but on what areas and for what purposes? (Is there a limitation on the employment of relatives in the organization?) Are there limitations on some types of expenditure or investments which may be contrary to the organization’s ethics?
Directors also need to know about the organization’s financial reserves. All organizations need to have safeguards against unexpected events; something to fall back on in a crisis.
Assuming that the directors do understand the financial papers, do all the key staff also share that same awareness? There is the risk that key staff who lack a financial background may not understand the information.
Financial Reporting
The reporting system here must include financial reports, a declaration by directors that the financial statements comply with accounting standards, that the financial statements give a true and fair view of the financial position, and there must be an auditor’s report.
The three key statements are Balance Sheet (a snapshot at a particular time), Profit and Loss Statement (history of revenue less expenses between the current Balance Sheet and the previous one), and Statement of Cash flows (history of cash in minus the cash out over the period between the current Balance Sheet and the previous one).
There should also be a cash flow budget indicating when cash will be received in the financial year and when the outgoings will be incurred so as to decide whether the organization can meet its debts as and when they become due and payable.
Directors also need to monitor the checks and balances that should in place to prevent errors, fraud and abuse. For example, board members should not use the organization’s resources for their own benefit. Directors should also check that the organization is adequately insured against all significant risks.
There is a fundamental tension in the arrangement of financial information. The accounting staff are producing reports for tax reporting and auditing purposes. But directors may need to have the information produced in a more reader-friendly style so that they can interpret it in the light of the organization’s strategy.
Another tension is that the reporting is done retrospectively (especially for taxation and auditing purposes) and yet the board has also to be forward-looking.
Monitoring the Organization’s Financial Situation
All NFP organizations are businesses. They may be “not for profit” but they need to operate on a business-like basis to remain in existence. This means making sure that the revenue base is secure (such as a steady rate of membership renewals or successful fund-raising projects).
Directors need to be sure that there is a correlation between the organization’s strategy and its financial operations.
Directors also need to ensure that all levels of appropriate staff have been involved in the budget’s creation. The “client-facing staff” know the basics of the organization and their knowledge should be sought. Creating budgets should not be left to the financial department.
Directors also need to ensure that the staff have accepted the budget. There is a risk in large organizations that staff make “wish lists” and that these get whittled down as the process continues up through the organization, with the final result as agreed by the board possibly bearing little resemblance to what the staff expected to be allocated. This lack of acceptance (“buy-in”) may result in little commitment by the staff to adhere to the budget.
The actual form of monitoring will vary from one organization to another. All that can be done here is to suggest a few matters that need to be considered.
If the organization receives a government grant, then care should be taken as to how it is recorded – is it for the current year or is it to cover the following year? (Government departments coming to the end of their financial year often try to distribute surplus monies rather than lose them back into consolidated revenue and so there may be a flurry of sums of money sent to NFP organizations towards the end of a financial year).
Under what conditions has the grant been issued? All grants carry obligations of some sort and they have to be adhered to. The grants could be for a capital purpose (such as a building) or for operational/ recurring reasons. When the grants are spent, they have to be accounted for (“acquitted”) with appropriate documents sent to the government department.
Legacies can be an important source of revenue (such as deceased members remembering the organization in their wills: “deferred giving”). The flow of legacies usually varies from one year to the next. It is probably unwise to put a figure in the budget for legacies (unless the organization knows for sure the amount that will come during the year). Legacies should be seen as the icing on the cake to permit expansion and new programmes – and not be relied upon as a hoped for regular source of operational revenue.
Donations may be easier to predict than legacies because there will some guide from previous years. An exogenous event may lead to a dramatic increase in one year (such as the impact of national tragedy on the work of a humanitarian organization and the public’s generous response to it).
Major variations in actual expenditure over the budget should always be investigated. They may be an early warning sign that something is going wrong (such as a sudden loss of revenue, an increase in workers’ compensation matters, or a sudden increase in expenditure).
Monitor the budget’s assumptions. All budgets contain assumptions (such as an increase in membership or a zero rate of inflation). The assumptions should be listed in the budget document at the beginning of the financial year and then checked on regularly as the financial year progresses.
Should the organization have stocks and shares? Some stocks and share may, for example, come to an organization via a donation or a bequest. They can be a volatile source of investment. At the very least, if the stocks and shares are to be retained, they should meet the organization’s ethical investment guidelines.
Making Sure the Organization does not Trade While Insolvent
Organizations must not trade when they are insolvent. Directors who allow an organization to do so face stiff criminal penalties.
All boards should have a system to monitor the organization’s financial progress so that enough warning is provided if a major financial crisis is looming and dramatic counter measures can be implemented (such as laying off some staff).
Among some of the warning signs are the following:
- optimistic budget assumptions are not being met (for example, an elaborate membership recruitment campaign failed to produce the anticipated number of members)
- what is the cash flow situation?
- problems with paying trade suppliers and other creditors on time
- problems of meeting loan repayment schedules
- is there too much inventory being held? (In other words, there has been a change in consumer/ member tastes)
- are there any large write-offs?
- is there a good debt collection system in place?
- how does this organization’s financial position compare to what is happening in similar organizations?
- is the organization postponing key expenditures (such as maintenance) to build up some cash cushion? Is key equipment not being replaced in time?
- are there industry peculiarities to which directors need to particularly sensitive? For example in some form of residential aged care centres, a resident will pay a market rate for home on the site; on the person’s death the deceased’s estate will receive much of that money back. This means that an aged care provider can have an apparently large sum of money on its books – but much of it does not belong to the organization.
THE WIDER NETWORK OF MONITORING MECHANISMS
Directors need to be aware that their organizations are being monitored in ways that may be not as obvious as the two formal organizational methods above.
First, all of an organization’s communications with its members sends an indirect overall message about the state of the organization. The quality and quantity of postal mailings and e-mailings all reflect the health or otherwise of the organization. This directors need to know what general information is being sent to members and the tone of the material. Does the material help maintain the brand? Is it sending out the right “message”?
Second, if the organization is providing some service to members of the general public, then the quality of the service will be monitored in some way. If the service is financed partly or in whole by government funding, then the government will have its own measures of assessment based on how the clients are treated. The organization measures its own performance in the delivery of services; here the government is looking at what the services mean from the client’s point of view of receiving the service. They may be different. Hopefully not but if there is a problem the government will let the organization know. Too many problems and the government will not renew the contract. It may also be difficult to acquire later contracts,
Third, there is the role of the staff as members of a trade or profession. Staff are loyal both to their employer and to their trade or profession; the latter usually being a longer-lasting loyalty. Thus childcare workers, for example, meet in professional associations and compare notes on how their respective employers are going. The directors and CEO may be making certain grand claims in public about the quality of their childcare service but these people may know from their own experience a different story. Word gets around – eventually perhaps back to a funding body or the media.
Finally, then, there is the even broader issue of reputation in the public domain, not least the role of the media. An organization may think it is doing a good job but an alienated former employee or a disgruntled client contacting a radio talkback station or a complaints-driven television programme, may well tell a different story.To conclude, directors need to be alert to the complexity of the measurement issues.
[1] The oft-quoted statement of Brazilian Catholic leader Dom Helder Camara is a useful warning here: “When I give food to the poor, they call me a saint. When I ask why the poor have no food, they call me a communist.”
[2] I served for a decade on the meta-data committee of the Australian Institute of Health and Welfare (AIHW). Australia is a world leader in medical data collection; AIHW has been working on a similar system for the welfare sector; it turned out to be even more complicated than the health task (http://meteor.aihw.gov.au)
[3] The most remarkable television example is the Seven Up series which has studied 14 British people since 1964 when they were all aged seven and checks in with them every seven years (most recently in 56 Up).
[4] Bob Garratt The Fish Rots From the Head: The Crisis in Our Boardrooms: Developing the Crucial Skills of the Competent Director, London: Profile, 2003, p 17-8
[5] As part of the growing professionalization of NFP organizations, many large ones are now creating in-house research departments. There is great potential here for organizations to produce reports not so much based on a scholarly review of the literature (which is already being done elsewhere) but speaking directly from what their own staff are saying; we need less academic debate and more grassroots assessments.
[6] J Gregory Dees “Enterprising Nonprofits”, Harvard Business Review, January-February 1998, p 57
[7] Quoted in his Guardian (UK) obituary. Marcinkus (1922-2006) handled the Vatican’s finances and was involved in two financial scandals; his full role was never investigated because the Italian courts ruled that Vatican employees were immune from prosecution. “Archbishop Paul Marckincus”, The Guardian: http://www.guardian.co.uk/news/2006/feb/23/guardianobituaries.religion/print