Data Collection
While data collection is critical for program evaluation, it isn’t the only function of your online learning program that requires reliable data. Most online learning programs are required to complete reports on a periodic basis regarding their operations. These reports might be required by state regulations or may simply be information desired by your governance structure.
Program marketing is another reason for collecting data. If you want to be able to promote your program’s successes, you will need to have the data to support your claims. Related to program marketing is the need to be able to provide data to support funding requests. For example, a state virtual school may find it useful to be able to provide personalized data to state legislators on how it is serving schools and students from the legislator’s district.
Yet another critical reason for collecting data is to ensure that your program does not develop any demographic biases. This is an example of data that is generally easier to gather during student registration rather than after the student has started or completed the course.
Understanding your data needs during program design is important because it allows for gathering the desired data in the most efficient manner possible. While data collection is important, you also don’t want to make the registration process overly burdensome or ask for data that is not likely to be known by the person completing the registration. As a concrete example, let’s take a look at a specific data collection issue based on the previous example of a state virtual school wanting to be able to provide personal reports to each state legislator.
Clearly it is necessary to know the legislative district of each student. However, asking for this information during registration probably won’t be effective because the person completing the registration process simply may not know the correct answer. So instead, it is important to determine what information can be easily collected during registration that will allow you to determine the legislative district. Perhaps your state department of education already has a database that allows you to look up the legislative district based on the school the student attends. If so, it is important to gather the school information in a way that facilitates this data look-up. Another option might be utilizing a commercial database that can determine a legislative district based on zip code. As you can see, the solution is not trivial, thus looking at these issues up front becomes very important if your program is going to be able to gather the data it needs.
As mentioned above, the completion of required program operation reports is one reason for data collection. These questions will help you determine if your data collection processes are adequate when it comes to reports that may be required for issues of access and equity.20
Data collection efforts go beyond determining what data will be collected at registration time or how the data collected at that time can be used to obtain other necessary data. It also includes the collection of data throughout all stages of your online learning program. Some examples of such data collection include:
- Data gathered from surveys
- students and/or parents about their experience in your online learning program
- teachers about their experience teaching for your program, often with an emphasis on how to improve the online content and/or instructional processes
- for supplemental programs, educators from the participating schools about their satisfaction with the program, as well as what additions and/or changes they would like to see to your online learning program
- Data gathered from teachers, especially data related to student performance
- Data gathered automatically by the Learning Management System
The data that is automatically captured by the Learning Management System has tremendous potential value for the improvement of instruction. The amount of time the student spends on each specific type of activity, as well as all of the digital communication that occurs, can be automatically captured, archived, and examined in efforts to identify what practices are most effective in achieving student success. At the same time, the amount of data generated often is overwhelming. Making sense of the data and drawing valid scientific conclusions typically requires individuals with expertise in educational research.
References for footnotes can be found on the resources page or click on the footnote number for a direct link.