Notes from Industry
In this blog post, we wanted to share some tips and best practices for non-profit organizations who are just getting started with data. Our goal is to help you get started with your data goals and to break down into small easy-to-do steps best practices to make the most of your data.
We are data and teaching fellows at Delta Analytics, a San Francisco Bay Area Nonprofit which has worked with over 50 nonprofits to date. Delta’s goal is simple – we pair professional data scientists, analysts, and software engineers in the Bay Area with nonprofits all over the world. Our data fellows are paired with nonprofit grant recipients for 6 months to leverage data to drive impact. We also build technical capacity in communities around the world by providing free trainings to help democratize access to machine learning and data tools.
One of our goals is to document some of the best practices that we share with our non-profit grant recipients. Throughout the years, Delta has seen many different hurdles that nonprofits face getting started with data – ranging from operating with limited technical resources to a disproportionate focus on populations and communities where existing data is limited.
This is a three part blog series: (1) Getting Started with Data, (2) Data Quality and Pre-Processing, and (3) Data Analysis. In this first part, we focus on helping you with:
- Framing a Data Question
- Figuring Out What Data You Need
- Getting Organizational Buy-In
- Calculating How Much Data You Need to Collect
1. Framing a Data Question
The first and most important part of starting any data project is framing the question to be answered.
This may require focusing on a specific area of your nonprofit’s mission or goals that you want to understand better or a question where you believe data will provide actionable insights. This could range from questions such as ‘how does your after-school educational program impact student outcomes?’ to ‘how effectively does your microloan strategy allocate its recipients with the funds they need to start a small business?’
Before you even start any data collection, it is critical to justify exactly why it is worth the effort of collecting data. Nonprofit resources are often limited, and having a clear and precisely-stated question avoids costly errors in data collection and duplication of efforts.
It is inefficient to allocate data resources to any project that will not find it necessary or impactful.
A few guidelines to keep in mind when framing your data question:
- Precision: Be as detailed as possible in how you frame your questions. Avoid generic words like ‘improve’ or ‘success.’ If you want to improve something, specify by how much. If you want to achieve something, specify by when.
- Data-Centric: Consider the role of data in your organization. Can data help you answer this question? Is it clear what data you will need to collect to answer this question? Can progress on the task be codified into a metric that can be measured? If the answer to any of these questions is ambiguous or negative, investing in additional data resources may be an inefficient allocation of resources.
Example:
Consider a nonprofit with a mission to improve literacy rates among children.
This nonprofit circulates children’s books amongst kindergarteners on a weekly basis in an effort to promote early literacy skills. The nonprofit wants to expand its operations and needs to increase the funding it receives from donations. The question that may immediately come to mind is ‘how can we increase our donation amounts?‘ At face value, this may seem like a reasonable question. But how does it compare to a question like ‘In the past 12 months, which of our interventions have resulted in the most donations, and how can we emulate their success to double our donations this quarter?’ Let’s assess these questions through the lens of our guidelines to find out.
2. Figure Out What Data You Need
We propose 4 guidelines for collecting the data needed in order to elevate any project.
Now that you have your question, you can begin to identify what data you need to make an actionable decision. Here are a few guidelines to keep in mind when deciding what data you need to answer your question:
- Necessity: Are the data you are collecting necessary? Avoid data bloat, which is over-collecting data points for a "just in case" scenario. This makes sustaining long-term data collection of those fields far more burdensome.
- Availability: Are there external, publicly available data sources like government data that you can leverage? If the data needs to be collected, how easy is it to collect? If it is hard to collect, do you have a plan and resources in place as to how you can ensure it is collected at regular intervals over time? One-off data collection is rarely helpful as there is no reference point to measure the impact of interventions over time.
- Maintainability: Can you maintain and easily update this data over time? Is the cost of doing so sustainable? This is critical because longitudinal data collection with standard fields is one of the most valuable resources for a nonprofit. Avoid constantly changing field names, a moving target of data collection objectives, and costly data collection procedures (like purchasing third-party data) that are not sustainable given your overall budget.
- Reliability: If you are using a third party data source, do you trust the quality of the data? What are the ways this data may be biased, incomplete, or inaccurate?
Example:
- Consider a nonprofit with a mission to find long-term housing for the unhoused. This organization may want to answer the question: ‘what percentage of the unhoused have we been able to successfully rehabilitate in the area we serve?‘. Let’s determine the data points that would be useful in answering this question:
3. Organizational Buy-in
Buy-in for any project at an organization is crucial to a project’s success.
Data projects take time and require organization-level buy-in. Make sure everyone on the team agrees on what data that you want to collect and measure and who owns the data collection process.
Example: Suppose teachers at a school are interested in fielding quantitative surveys to track student outcomes, but there exists little incentive for teachers to collect this data on top of regular work. As a result, only one teacher in the school volunteers to design and administer the survey to their class. However, the survey results will now be limited to the students’ experiences and outcomes for just the one class. The measured outcomes will be biased because they will not capture any variance across students from different classes in the school.
Since collecting data can be a time-consuming and resource-intensive process, it is worth the extra effort to involve all teachers to administer the survey to each of their classes and have regular check-ins to ensure the success of the project.
4. Calculate How Much Data You Need to Collect
Resource-constrained nonprofits may benefit from thinking ahead about how much data to collect in order to conserve their resources.
Gathering data to aid decision-making is often more of a luxury than a core capability. For data that is not readily available and needs to be collected, acquired, or purchased, it is important to assess how much data is needed to proceed with the task at hand.
Example: Designing and fielding surveys can be time-consuming and resource-intensive. Following from our previous example, the teachers can be strategic about how many survey responses need to be collected to conduct quantitative analysis of the survey results. In such cases, one can compute the required sample size. SurveyMonkey provides a helpful calculator here.
Summary
In this blog post, we introduced key principles to consider as you begin collecting data for the first time.
This concludes the first part of our three-part blog series. Stay tuned for Part 2 – Data Quality and Pre-Processing!
If you are interested in additional blog posts on this topic, please reference our two-part blog series on how social sector organizations that are getting started or reevaluating their data efforts can more effectively integrate data with their organization and mission:
Authors & Reviewers: Hani Azam, Karun Singh, Allie Wang, Raul Maldonado, Sara Hooker, Amanda Su, Melissa Fabros & Sean McPherson
