Demonstrating Impact
To help get you started, please consider the following questions related to this topic:
- In what ways can service produce the greatest impact?
- How can the Corporation for National and Community Service better demonstrate the impact of service?
- What is the best way to identify program models that work?
- How should the Corporation transition from the current practice of self-nominated performance measures in AmeriCorps to standardized measures? For example, the Act sets forth standard measures for each of the five Service Corps. In education, the legislation includes performance measures such as: student engagement, student attendance, student behavior, and student academic achievement. What support do grantees need to implement these new measures?
- Should the Corporation establish standard national performance measures in VISTA, Senior Corps and Learn and Serve? If so, how?
- Should there be a few measures that apply to all Corporation programs?
- Other
Disclaimer
Except where specifically noted, any views or opinions
expressed in this forum are those of the individual contributors. The views
and opinions do not necessarily reflect those of the Corporation for
National and Community Service or the Federal government. Please
click here to read the discussion board policy
for this site.
Excerpt from Comments Provided by CaliforniaVolunteers
The Corporation should work with the field to ensure that it is capturing meaningful data and doing so in a way that is not disruptive to the operation of its programs.
Rule Content
· Transition to standardized performance measures: Adoption of standardized measures should be gradually phased in so that the requirement would immediately take effect for new programs or those recompleting for funding, not for those grantees that are in the second or third year of a grant cycle. This would allow the Corporation and state commissions to provide stronger technical assistance, and would ensure that current grantees can effectively transition to the new measures.
Rule Development
· Consult with the field: As the Corporation considers standardized performance measures, it should do so in consultation with grantees and experts in the field. This might take the form of a working group or advisory committee composed of performance measure specialists, grantees from all streams of service, and state commissions with experience in developing common performance measures.
· Focus on common activities: Standardized performance measures should focus on activities common to all grantees in a given program. For example, a common measure could focus on recruitment and management of volunteers, which is a requirement for all AmeriCorps programs.
Inclusion of People with Disabilities in National Service and Volunteerism
Corporation grantees have requested a standardized tool and related methods to accurately identify the number of people with disabilities serving. By consistently determining the prevalence of members and volunteers with disabilities, CNCS grantees and CNCS can accurately measure an indicator of progress, from state to state. Grantees would also benefit from a common and universal definition of reasonable accommodations, a consistent method to collect data on requests, the on-going process of the provision and evaluation of reasonable accommodations.
Tools and methods used for prevalence and accommodation measurement should be respectful, value diversity and developed with the expertise of the disability community. The implementation of standard methods to identify numbers of people with disabilities serving and accommodations would provide an accurate and defensible portrayal of progress toward inclusion.
The House That Candy Built
Other- A Story About Community Service in America
Two years ago, I heard the houses on Feazell Street, near Stephen F. Austin in Nacogdoches Tx, were going up for sale soon. I was shown the houses on the road, and immediately fell in love with 119 Feazell, a house apparantly built in the 50s, with two dormer windows. I asked about the house, and was told it was sold already. I asked again every two months. Only once was I told they were bulldozing every house on the road, and I asked them to reconsider. For two years, we bugged the housing dept, and left phone numbers in case there were any changes.
They had a meeting and decided to give the community a chance to bid on and move the houses. 6 or 7 houses, sealed bids, and every one sold. Every one who bought a house, including me, was given 30 days to move the house. I decided to sell pralines and divinity and name the house The House That Candy Built. The local newspaper ran the story of the house that would be bulldozed if it wasnt moved in thirty days, naming the resaurant that agreed to sell the candy on their counter, Country Kitchen, my daughter and freind set up a website. The computer crashed. The candy I sold for a dollar was first bought at the colllege where i bought the house. Then we told the banker what we were tying to do, and he gave five dollars for a dollar praline. A man read the story and donated a hundred. Another lady said God told her to donate 500 for a computer, and she did. At first Country Kitchen was paying for the candy and selling it, but they quickly decided that wasnt working, it was selling too fast. So they put up a jar on the counter, and let people take care of it. After that, they donated their change, and gobbled up the pralines so fast I had to deliver them twice on several days. A local business with a no solicitaion policy read the story at work, and so many people showed interest I was asked to make a delivery there. The candy sold so quick there, that I donated the next batch to the people who didnt make it in the first five minutes the first time I went. A lady donated 30 there and wanted no candy. My daughters in the Navy in Japan sent money. My uncle living halfway across the country bought two batches by mail. The house was moved within the thirty days, the house mover donated the blocks it sits on. Never have I seen a community show as much support for a project as this, and though its taking a while to rebuild, the roof has almost been rebuilt now. A hundred was donated by my cousin for paint, which sits waiting for the house to be scraped. The two upstairs rooms and bathroom have been framed in, and the roof covered with plywood, with only one dormer window left to have it back in the dry. A church group emailed offering to have a bake sale, tear down any room that needed torn down. A civic group offered to rebuild anything we had to tear down to move it, and I will call them when its ready to paint. I'm just a woman who fell in love with her dream house. With the help of the town of Nacogdoches, and some candy, this house will never be touchd by a bulldozer. So many people helped to make this happen. SFA, even deciding to give us a chance on this was a miracale. As I sit in my one bedroom house with two kids, I consider all the miracles that made my dream house next door be moved from Feazell to this spot. A house mover who I will not name, charging 18000 to move the house down the road, 8500 to move mine. And I havent read the Act that suggests Community Service, but community service is the only thing that kept this house from being bulldozed. Prayer and a bunch of people. On Feasell Street the lots are cleaned up, all houses that could be moved have been moved. To me this story deserves to be told. Maybe it will make some school give someone else a chance, however slim, to bid on and move a house thats destined for the dozer. Maybe it will make someone who's fallen in love with a house try just a little harder to change its destiny. Teachers bought candy, and I gained 20 pounds just testing all the candy , one peice of each batch. And I was a 95 pound woman who tried for years to gain 20 pounds. I'm now trying to lose ten rebuilding the house, pulling nails from one by fours, scraping it, but the paint is there if I ever finish. Comments and Questions welcome.
Excerpt from Recommendations Developed by the Maine Commission for Community Service (MCCS)
Leadership Opportunities For CNCS In The Field Of Community Volunteer Service
#1. Continue to use the “bully pulpit” on the national scene to foster among nonprofits a respect for the value of volunteers and a commitment to providing resources that will ensure success in volunteer endeavors.
#2. Be clearer and stronger in articulating the ties between all volunteer service programs supported by CNCS and implementation by that program of the essential volunteer management practices (22 traits of high quality programs).
#3. Target the Social Innovation fund at the causes of community problems, not the symptoms.
Parameters For The State/Federal Partnership
#4. Conceptually sort the authorized CNCS programs into two categories with two different sets of outcome, application, and reporting expectations.
#5. Whatever the appropriation for the Volunteer Generation Fund, maintain the 50% formula to State Commissions and 50% competitive allocation from the beginning.
For years, rural and small State Commissions have been cobbling together bits and pieces of resources to try and help volunteer centers, organize days of service, and promote professional development among volunteer managers.
Setting the precedent of giving states dependable rather than sporadic resources is essential and responds to the hope that has been raised in our partners who have been reading the news closely. This precedent needs to start in the first year, not a subsequent year as proposed.
Include impact measures related to increasing the capacity of a participating local agency to provide services to their community.
As required by the Older Americans Act, the US Department of Labor has conducted three annual independent surveys of local nonprofit and public agencies that “host” participants in its Senior Community Service Employment Program (SCSEP). A national sample of over 10,000 agencies was asked, “How has your agency’s ability to provide services to the community been affected by its participation in the Older Worker Program?” The most recent survey of data collected through February 2009 indicated that 44% of the agencies reported that their ability had been “significantly increased” and another 31% reported “somewhat increased” (for a combined total of 75%). The Corporation may wish to explore possible lessons from these surveys for its own impact studies. An interagency agreement between the Corporation and USDOL could facilitate sharing of methodologies for measuring impact and other joint efforts.
Performance Measures
CNCS should require certain performance measures and organizations should be held to them. However, it would be helpful for program improvement to allow "experimental" performance measures as well that provide information, but organizations shouldn't be obligated or penalized for the outcomes of those unless they indicate a serious problem in the program or its management. This would allow for program innovation, expansion and improvement.
Longitudinal Study
The Mississippi Campus Compact strongly supports conducting a longitudinal study of service-learning and its impacts throughout education. We ask that CNCS make this a top priority during this time of opportunity and growth so as to build a stronger and sustainable foundation for service-learning in the future.
Demonstrating impact of service-learning/engaged learning
That the Corporation's service-learning grant program progress reports and terminal evaluation (and perhaps grant program proposals) are so highly quantitative seems to be a problem. I am advocating here for an inclusion of **qualitative** data in proposal and program evaluation. However, we know that self-reports from students can very easily turn to baloney, because students, particularly high school and college students, are very well practiced in "writing for the teacher." They know exactly how to intuit what teachers or other authority figures want to hear and tend to produce exactly that. (This is one reason why our undergraduate engaged learning program is suspicious of "reflection" essays as evidence of learning.) So what to do? I am not an assessment expert, but I would like to offer some suggestions:
Leveraging Social Media to Demonstrate Impact
Demonstrating the impact of service is a crucial element to the implementation of the Edward M. Kennedy Serve America Act. One of the most effective ways to record and communicate the impact made by service participants is through self-reported stories of their experiences. When linked to other quantitative measures, this means of tracking and measuring impact becomes more powerful and valuable.
Qualitative reporting of impact is necessary to fulfill various requirements of the legislation, including “the change in attitude toward civic engagement among the participants and the beneficiaries of the service.”
The Corporation already recognizes that structured reflection is critical to developing knowledge, skills and judgment of volunteers. In addition, if such story telling is accomplished through the use of social media, it will also serve to inspire more citizens to become involved in service and to grow the movement that the Serve America Act recognizes.
In the same way that it might move toward a uniform way to measure, track and demonstrate impact through quantitative metrics, the Corporation should consider providing a framework for its programs and grantees to utilize social media tools for the same ends.
During this process, it is essential to consider tools that already exist that can be applied in a way to demonstrate impact, which simultaneously allow for a greater degree of accountability and transparency. Existing social media sites and tools can be leveraged in powerful and engaging ways, especially if the content within is aggregated into a single platform or social network specifically designed to national service programs. This may also extend to other organizations doing complimentary work but not under the explicit direction of the Corporation.
The demonstration of impact through social media will provide many benefits and should be a core focus by the Corporation in implementing the Serve America Act.
Other--Tribal and Higher Education
Clearly Service Learning is very different than AmeriCorps programs.
Tribal--requires cultural and tribal consideration when a program is being established, evaluated or expanded. Standardized performance measures does not fit with the concepts of tribal programs based on culture or language preservation. I would strongly suggest that all tribal program grantee identify the measures they consider that indicate success, areas in need and impact. Important note: when working in Indian Country we do not approach others or program in a linear manner, but in a circular process--holistic in nature.
Tribal, Territories and Higher Education should establish any standard national performances measures within Learn and Serve.
Tangible and Intangible Results
With the proper investment and support, costs of volunteers and volunteer programs are repaid many times in both tangible and intangible ways. While we must seriously measure and be accountable for the tangible results, the intangible results of volunteer efforts will also have an enduring affect on individual lives and communities. Thought should also be devoted to ways to think about and discuss the intangible results of volunteerism.
standard measures
I fully support implemetning a set of targeted standard measures across programst=. Its is very difficult to "roll up" and demonstrate community-level change if you are not collecting standardized information. Of course, programs should also have the opportunity to measure against other, more program-specific results.
I suggest you take things one step further. Not only should all Corporation programs use standard measures, but I suggest CNCS partner with national groups across the country to work to encourage nonprofits to collaborate on adopting and reporting on these measures. For example, I work for a local United Way, and we focus our work on education, income and health. It would be extremely powerful if the thousands of UWs across the country agreed to use the standard measure related to those areas. CNCS and UWA could be collaborating to report to the country on the amplified impact their work is having on student academic achievement. Many of the national organizations are doing complimentary work -- it would be very powerful to bring this work together.
Demonstrating Impact - Essential but Not Always Equal
Each AmeriCorps program has to describe their impact of service. It's a federal requirement, plus State Commissions, the Corporation, and Congress likes to see whether or not federal AmeriCorps funding is showing a return on their investment on top of the valuable community service being provided. It is a fine balance of bean-counting (quantitative) and community and program satisfaction (qualitative) of service provided.
Here are a few observations from a State Commission perspective:
Some AmeriCorps subgrantees write fantastic performance measures with a firm grasp of the logic model and analytical thinking involved in management, accountability and performance principles with the service being provided, but run a service program with some challenging issues. Progress reports are detailed, overly analytical, and you really have to hunt through the analysis whether true impact was really achieved and was likely completed over the course of a couple weeks.
Some AmeriCorps subgrantees write mediocre performance measures with a working knowledge of the logic model, only because they are required to do so for their grant submission requirements, but run a pretty credible AmeriCorps program with a strong reputation locally and sometimes nationally. Progress reports tend to be simplified, strong on the outputs low on the outcomes, and likely completed over the course of a few days.
With this said, it is safe to say some AmeriCorps Programs are good at demonstrating impact and some are not, or at least some Programs think they write good performance measures. It is also safe to say some State Commissions are better at reviewing, commenting, and approving strong performance measures than others. Project STAR has done a stellar job of educating both Commissions and Programs about good performance measurement practices, but not all Commissions or Programs have staff that specialize in evaluation, logic model, data collection tools, statistics, regression analysis practices, and so on. It's like a right brain/left brain dilemma and where some Programs and State Commissions tend to put their focus to.
The new Progress Report implemented by the Corporation is straightforward. You list the performance measure - then describe whether it was met or not or ongoing. However, it is tough to determine the true impact if the performance measurement used or data collection model was weak to begin with.
In such cases above, good Standardized Performance measures can be used as an option. There could be standardized measures for items like Education (students reading at/above grade level, test scores, student attitudes, teacher satisfaction, etc.); Volunteer Management (number recruited, number likely to continue at site, % to continue after program year); Disability Inclusion (fill rates/retention); and Member Development (number trained, % who feel will use training in future career). CNCS could use tried/tested performance measures as identified by State Commissions and CNCS Program Officers, and put them out there as some boilerplate models that programs and new applicants could use with some slight modification.
State Commissions who have a comfort level in reviewing performance measurements, can make constructive comments on how well they measure impact, and feel good about the decisions that led to their approval, then I would allow the Commissions to move forward as they usually do. I agree with a previous comment, that having too many standard measures might hinder innovation, but these standard measures can help a program who struggle in this area and give time for a State Commission to provide ongoing help and support.
Perspective from Volunteer Resources Managers regarding Demonstrating Impact
· Both statewide and local program models that demonstrate best practices in volunteer resources leadership and results for identified indicators should be publicly highlighted and documented with lessons learned. Related information should be disseminated throughout the nation.
· Existing successful programs and initiatives at the state and local levels should be reinforced with resources for replication, and new programs and initiatives should be initiated in areas where volunteer programs do not yet exist.
· Volunteer resources leaders who are experienced state and local practitioners should be an integral part of designing evidence and evaluation indicators and assessing success.
· Volunteers also should be a part of the above process.
· Sate and local indicators of success, designed locally and at the state level, will lead to greatest results.
· Volunteerism by nature, as a part of the democratic process, should be driven to a large extent by the people carrying out and leading the efforts on a day to day basis.
· Experienced state and local volunteer resources leaders have been greatly underrepresented in the Corporation’s leadership design and efforts. Representation from this group should be significantly increased to bring this special expertise to Serve America Act efforts and to parity with other representation.
How can the Corporation for National and Community Service better demonstrate the impact of service?
When we measure service in terms of hours, we disallow or at least discourage deliverable projects that cannot be measured in hours. Doing so hoards the resources of higher education and reinforces the impression that service-learning is designed for placements that are integrated primarily into health professions, social sciences, and humanities. For example, a mass comm class might produce a public relations campaign for a non-profit or an engineering class might design and build a playground for children with disabilities. Faculty who develop these kinds of service-learning projects resist efforts to quantify the work. The quality of the deliverable itself and the community partners' assessments of whether it meets their needs makes more sense to most faculty and students.
What is the best way to identify program models that work?
My organization is the California Administrative Office of the Courts, a California Volunteers statewide AmeriCorps grantee administering and managing the California JusticeCorps program. We feel there should be staff at CNCS dedicated to identifying and fostering local and or state level initiatives that have promise for national expansion. State level commissions can help to identify well run and strong performing local programs that could be replicated in other states to become nationwide and can communicate that to designated staff at the national level. Existing national programs like City Year and Teach For America, for example, have evolved from local to nationwide in a variety of different ways and use different administrative models, multiple funding streams, etc. There is not one "path" to national expansion or even easy access to technical assistance needed to stratagize a national expansion for everyone's benefit - the members, the target populations being served. We believe that JusticeCorps, and other promising programs out there, have the potential to grow nationally. But we really don't currently have access to the technical assistance and strategic advice to do so.
Agencies should measure effectiveness as results per $
Results should be the focus of the work, but more important the cost of achieving these results should be paramount.
Demonstrating the Impact of Service
The challenge of demonstrating impact of service is two-fold. First, not all service is equal. Service-Learning arguably exerts a much greater impact than service alone. Academic Service-Learning targets course curriculum outcomes (discipline specific content), general learner outcomes (universal outcomes such as diversity, interpersonal, intrapersonal, and critical thinking skills), community partner outcomes (increased capacity), faculty outcomes (teaching tool, personal growth), educational institution outcomes (community outreach, lifelong learning). Service-Learning that involves a real connection with the population served and a reflective component produces much greater results than service alone. The second challenge in demonstrating impact is the lack of standardized instruments. Building reliable assessment instruments is a science and requires the skills of social scientists. We currently have a plethora of poorly constructed instruments in the field of service-learning that depend largely on self report data. We need to invest resources into instrument construction. The team must consist of social scientists who have research and statistical training in addition to a thorough understanding of service, service-learning and academic service-learning.
Move away from the piecemeal approach
To help us understand the real impacts of service on participants, communities, and institutions, we need to have a more planful and articulated approach to the study of community service (and hopefully service-learning, too!). The research on community service and volunteerism is all over the place, as attempts have been made to measure every possible participant outcome under the sun. We seem to just pull instruments off the shelf and apply them loosely without really thinking through the purposes and populations for which they were designed. We set very lofty program goals (reduce homelessness, improve literacy, etc.) that the average community service program is unlikely to ever accomplish, and then we have to try to prove that community service is effective! Good luck!
I would like to see a more systematic approach to the study of community service that focuses on answering questions that have been asked over the years in the dozen or so research agendas that exist in the field. To really build an understanding of community service, we need to conduct multiple investigations on an issue, build on prior research studies, use well-tested instruments in appropriate ways, apply mixed methods and triangulated data from multiple sources, and focus on investigating outcomes that are reasonable and measurable.
The research in the field remains too scattered (despite many calls for better coordination), making it almost impossible to conduct meta-analyses or to generalize study findings beyond the particular programs studied.
A more thoughtful and planful approach to the study of community service, even if it emphasizes a narrow set of issues in the field, can garner us many moreuseful insights that will provide the kinds of evidence we need to make the case for community service investments and programming.
Serve America Act Implementation
AmeriCorps has been the driving force of my career and my life, and many others’ lives as well. Equal Justice Works manages a national AmeriCorps program for Lawyers and law students. Our office is in Washington DC, but this summer 420 law student AmeriCorps members will serve in Louisiana and 44 other states.
The best example of Equal Justice Works AmeriCorps and its impact is the Katrina Legal Initiative. Since Katrina hit, Equal Justice Works has placed 20 AmeriCorps Legal Fellows and over 100 Summer Corps members in Louisiana, Alabama and Mississippi to respond to the legal needs of the Gulf Coast Region. Summer Corps members Haim Vazquez Echeverri, Sai Lui and Robert Whiting are among the first AmeriCorps members serving through the Recovery Act. AmeriCorps Legal Fellows Amanda Furst and Amanda Golob, will serve as AmeriCorps Recovery Fellows next year, to help people access health care and preserve their homes.
The Serve America Act takes great strides toward easing the financial and regulatory burdens of individuals and organizations to participate in AmeriCorps. Through legislative interpretation and rulemaking, we must make the most of this opportunity to make it easier to get things done for America. Equal Justice Works funds organizations which receive grants from other Federal agencies, including HUD, HHS, the Veteran’s Administration, and the Legal Services Corporation.
In working with local partners, I hear two things consistently: First, CNCS staff is the easiest to work with. Second, AmeriCorps funding requires the most reporting and restrictions for the least amount of funding. The disproportionate burden on grantees is disturbing. Also, people fail to take into account the impact this has on AmeriCorps members, whose hosts may fail to support them adequately due to their paperwork burden. Through legislative interpretation and rulemaking, we must make the most of this opportunity to make it easier to get things done for America. Some initial suggestions:
1) Fixed Price grants have to be made to work for mixed (FT/EAO) programs. These models have proven impact and efficiency, and should not be excluded from the trend toward fixed price grants. A simple solution would be for programs to determine the costs of each type of slot in their grant applications.
2) Common performance measures are important to demonstrate national impact, but they create other problems. Standardization kills innovation. At a time when we’re looking for social innovation and entrepreneurial programs, why create limits? In Montana, we have an AmeriCorps member who helps Native Americans with their wills. In Detroit, AmeriCorps Fellows will serve victims of domestic violence. Here in New Orleans, AmeriCorps Legal Fellows Amanda Golob and Amanda Furst help victims of Katrina access healthcare and maintain their housing. We struggle with common PMs in one program. This will be challenging to implement across the country.
3) With all of the attention and funding slated for AmeriCorps, Equal Justice Works is concerned that AmeriCorps living allowances remain tied to the poverty line. The poverty line itself is widely regarded as outdated, and significantly lower than it should be. Indexing the Segal AmeriCorps education award to the level of Pell Grants is a powerful step, but some of our AmeriCorps Legal Fellows have $120,000 in school debt. We all agree that service involves sacrifice. Too often, the commitment to serve places people and families into economic jeopardy. Just $5,000 more per member would demonstrate a meaningful commitment to those who serve. We can do better.
Equal Justice Works is extremely proud of our 15 year history managing AmeriCorps programs, and we look forward to working with CNCS to address the economic crisis and expand opportunity for all Americans. Thank you for the opportunity to weigh in on the Serve America Act.
The greatest impact is through capacity building, local recruiting
One of a VISTAs primary duties is to build capacity, and after nearly two years as a VISTA I think that's really the most important role we can play in a community. Many poor areas don't have people with the skills to write grants, organize people and build community support. Training and placing VISTAs in their home communities is one way to do this. Another is for VISTAs -- no matter where they are from -- train and support local volunteers.
Four years ago when a VISTA project started in my hometown, a local guy, a recent college graduate, was placed as the VISTA. There was no organization for him to serve, so his project was run out of a university 100 miles away. After two years, he had created a volunteer base and incorporated a local citizens group. That group was still inactive but it was here. I am also local, and when I took over as VISTA there as a very short learning curve. A year later the organization that the first VISTA founded is a 501 c3 nonprofit corporation; it has an active, five-member board; it has become a player in local decision making on the environment and development; it is transitioning to a local supervisor for its VISTA this year and will take on a second position through the Recovery and Reinvestment Act next month. It has done environmental projects, economic development projects, and informed the community about things that are being done and things that still need to be done.
This is real impact on this community, and a model I think VISTA should strive to follow.
Greatest Impact
Does National Service priorities reflect the immediate and future needs of the community it serves? Is there local research to substantiate need?
In preparation for submitting a 3 year Senior Companion Program grant proposal local and national research confirmed the efficacy of low income seniors providing in-home assistance to frail adults. It underscored: the increasing community need for one-on-one care that enables an individual to age in place; the physical, emotional and financial toll of care giving relieved by respite care; the positive rewards of volunteering as a Senior Companion; and taxpayer savings comparing Senior Companion stipends/travel to the cost of Medicaid paying for long term care.
Provide inter-state forums to highlight best practices via teleconferencing
Demonstrating Impact
In my opinion, the Corporation should be expansive in the ways that they consider impact, in terms of who is being impacted, the types of impact that service has had, the context under which impact is maximized, and whether the impact is sustained over time. The "who" should include both those being served and those providing service, along with impact on the institutions and communities in which they operate. The types of individual impacts should include cognitive, behavioral, social, emotional, economic, and aspirational impacts for both those providing service and those being served. Institutional and community impacts could measure changes in culture, infrastructure, economic viability, and "quality of life." The Corporation should triangulate its data, emphasizing as much as possible the "hard" data - acquisition of knowledge, skills and changes in community indicators. Counting hours and people is not enough -- measuring specifically how the programs made a difference would be better.
Greatest impact can be discovered through research. A prime example was in the way that the K-12 standards and indicators of quality service-learning practice were developed. The standards and indicators came from the literature on outcomes; variables were investigated as moderators or mediators of impact. The same could hold true for the AmeriCorps, Senior Corps, VISTA, or NCCC programs once desired outcomes were documented and places with the highest impact were studied. In the case of the K-12 standards, a traditional standards-setting process was used. In this case, the idea was to maximize impact so that the case could be made for service-learning in schools. Why not use a process like that?
There is much more to be said and we in the field particularly appreciate the solicitation of input.
Shelley H. Billig, Vice President, RMC Research Corporation
What is the best way to identify program models that work?
The best way to identify program models that work is to identify exemply service-learning schools and programs. Why not re-institute the "National Service-Learning Leader Schools" program? The Corporation had a competitive/recognition program called, "National Service-Learning Leader School". Two of our district schools in Hudson, MA were named as "National Service-Learning Leader Schools" first one in 1997 and another in 2002. We continue to serve as models of service-learning for other schools in Massachusetts and across the United States.
What is the best way to identify program models that work?
There are some terrific models for assessment that have been developed by the non-profit sector already to measure impact and we urge CNS to look at those models when considering the criteria for evaluation and assessment. Americans for the Arts through a project sponsored by the Kellogg Foundation recently concluded two years of work focused on the question of assessment and how to design models that accurately capture the social and civic efficacy of arts-based civic engagement. Through this work, Americans for the Arts partnered with exemplary, community-based organizations, collecting and analyzing metric and other evaluation models that can help arts practitioners, as well as funders, measure the social/civic impact of their work in more concrete terms and thereby enhance the case for the arts’ value and contribution in civic engagement. A small working group of funders and practitioners worked in tandem with experts in social or civic metrics/evaluation to advise and support these efforts.
Standardized Performance Measures
The Georgia Commission for Service and Volunteerism supports the concept of creating some standardized performance measures for AmeriCorps programs. Our commission asks CNCS to consider a combination of 1)standardized and 2) self-nominated performance measures, along with the option 3) for state commissions to institute their own required performance measures.
Our commission suggests that the benefits of such a combined set of performance measures would be increased accountability, increased ability of both CNCS and state commissions to demonstrate program impact, and retained flexibility of sub grantees to design and then measure programming that meets their local needs. New programs would especially benefit from this newly structured set of performance measures because part of the work of creating this important program component would already be done for them.
As CNCS and state commissions face the challenge of preparing for the programming growth projected by the Serve Ameica Act, our commission suggests that CNCS consult with commission and current program staff on appropriate performance measures that can be standardized for the five new Corps service areas as a starting point.
Uniform standards that apply to all programs
One standard that should apply to all programs is cost/benefit (cost of the program versus the total benefit it provides). This would seem a fair criterion for continuing to fund programs and prioritizing funding.
The more you can roll up your performance measure info, the great the numbers are.....: however, you loose the stories and individual
impact. You also may gloss over local impacts and as we all know, we seem to all measure things differently....oh joy!
Talk with Washington Service Corps and Project Star, regarding their transition over the years to program nominated performance measures to common measures.
The past few years all programs have had the same 4-5 goals: it has not seemed to help collaboration and partnership development and
impact measurement. We get very few AC/VISTA programs using service learning as strategy for youth engagement.
If you want certain impacts, it seems you would have standard national performance measures: this will be fun if you are trying to maintain the diversity of your programs.......
Welcome
Welcome to the "Demonstrating Impact" discussion forum. We look forward to your thoughts and ideas!