WASHINGTON – The U.S. Department of Commerce and the U.S. Small Business Administration, on behalf of the Evaluating Business Technical Assistance Program working group, today released a report on “Building Smarter Data for Evaluating Business Assistance Programs – A Guide for Practitioners
”.
The report is authored by program practitioners, economists, statisticians, and evaluation and policy staff from the U.S. Department of Agriculture, U.S. Department of Commerce, U.S. Department of Labor, and the U.S. Small Business Administration. It takes a practitioners view and identifies critical data and best practices that support the use and improvement of program administrative data and other existing data sources for rigorous impact evaluations of federal business assistance programs.
Key points of the report include:
· Agencies that provide business assistance compete for scarce resources. Managers need findings from well-designed evaluations to support assertions about program impact, and demonstrate that a program provides a better return-on-investment than alternative approaches.
· Data from federal business assistance programs can be used for rigorous assessment of program impact, when combined with other sources of government or private data. The combination can provide the necessary information on both assisted businesses and similar businesses that did not receive assistance to analyze whether the assistance made a difference in business results. The combined data may also permit the evaluator to expand the scope of the analysis to include more years and consider additional research questions. In essence, this approach can reduce evaluation costs, increase quality, and not add to survey burden on businesses and taxpayers.
· When programs are launched, the information and protocols that facilitate linking program data to other data sources should be anticipated. Lack of this advanced planning has been one of the greatest impediments to using existing datasets effectively for impact evaluation. This guide addresses this recurring problem and recommends that program managers:
o Identify administrative data needed for both program service delivery and eventual impact evaluation at the beginning of a program or pilot. This prevents the need for expensive, after-the-fact additional data collection. Similarly, for existing programs, early assessments of the quality and availability of administrative data and actions needed to remedy data deficiencies can increase the value of administrative data for eventual evaluation.
o Solicit the input of evaluation experts early in the process of developing data plans to identify the best methodology for measuring program impact and the most cost-effective ways to assemble the data necessary to support high-quality evaluations.
o Explore whether linking program administrative data on assisted businesses to other data sources is a viable option. This linkage, which requires sufficient unique applicant/participant-level identifying information in all datasets, can increase evaluation quality and reduce the need to conduct post-service surveys. Sufficient privacy, security and confidentiality procedures must be in place.
o Engage departmental attorneys and policy officers, including privacy, confidentiality, and security officers, early in the process of developing data plans. This can avoid problems and delays that arise when data collection and sharing for evaluation purposes are treated as separate or after-the-fact considerations.
The entire report can be found online at: https://www.sba.gov/sites/default/files/aboutsbaarticle/Building_Smarter_Data1.pdf