Remediation is Not a Barrier: Confusing Causation with Correlation
by Alexandros M. Goudas January 2017
If asked, I would be the first person to proclaim that postsecondary remedial graduation rates are too low. But I also seem to be the first to point out that these rates are completely normal: They are exactly what we should expect within the context of all US college graduation rates. In other words, once factors such as minority status, part-time status, and open enrollment institution type are considered, students who take remedial courses have very similar chances at graduating when compared to similar students who are not required to take remedial courses. This does not suggest that remediation is holding students back more than any other potential risk factor.
Unfortunately, however, for nearly a decade now the idea that remedial courses are a barrier has taken over higher education. Many educators, news articles, policy experts, advocacy groups, research centers, and legislators repeat this claim to the point that no one seems to question it. And because this sentiment is becoming accepted as true, even by some educators in the field, remediation is being restricted, excessively reformed, or cut entirely. It is clear that the data underlying this claim must be explored further. Without this analysis, we run the risk of claiming that since remedial students take remedial courses first, and these students have low graduation rates, it must be because of those remedial courses. This may be confusing causation with correlation.
First, we must define what a barrier is. Many reports claiming that remediation is a barrier cite low numbers of success and high rates of dropouts. A few of them can be read here, here, here, and here. However, very few of these reports include any comparison data. For example, the first link states, “Just 16 percent of students placed in developmental education earn a certificate or associate degree in six years” (par. 3), but it doesn’t cite what percent of similar nonremedial students, according to age, race, income level, full- or part-time status, also graduate in six years. Is it 18% or 30%?
The middle two links fail to refer to any comparison data as well. However, in the last link, the Community College Research Center states, “Only 28 percent of community college students who take a developmental education course go on to earn a degree within eight years” (Jaggars & Stacey, p. 1), and in an endnote, they cite a comparison figure of 43% for nonremedial students (p. 6).
In order to figure out if 16% (in six years) and 28% (in eight years) are abnormally low numbers, we must know what baseline or comparison data exist, in addition to the 43% number. If it were true that remediation is a barrier, then it would also be true that only remedial students as a demographic would have low completion rates, and all other groups would have much higher rates, because as a barrier, remedial courses would be holding their students back almost completely. We should look at some more completion rates in context, then, to see whether this indeed is the case.
The best graduation data currently comes from the National Student Clearinghouse Research Center’s Signature Reports (NSCRC). The reason why it is the best data is because they track students when they transfer and complete at other institutions, and they track part-time students as well. The National Center for Education Statistics’ IPEDS data, by contrast, typically only tracks first-time, full-time (FTE), degree-seeking students. At community colleges, this may exclude up to two-thirds or more of all students, many of whom are remedial students. The NSCRC cohorts number in the millions as well, they are independent and a nonprofit, and other reputable organizations cite them regularly. It can be considered reliable data.
The NSCRC’s latest longitudinal report (2016) tracks cohorts of students who first enrolled in any institution type in the year 2010. It is the fifth annual report of its kind, and one of the best parts about it is that it uses the same metrics, and even the same figure numbers, so that one can easily analyze changes over time. It also uses six-year outcomes, which means it tracks all students for six years, instead of three or four, so that a more accurate picture of graduation rates can be understood. As we know, only a small percent of students actually graduate within the nominal time frames (2-year degree in two years; 4-year degree in four years), so calculating graduation in a 150% of these times is conventional now.
Looking at Figure 19 in the NSCRC’s 2016 report, we can see that indeed there is a distinct gap between the completion rates of part-time students and full-time students in two-year public colleges. These numbers have not changed much in the five annual reports, so it is safe to assume that these numbers could be reliable baseline data.
What this means is that it is completely normal for exclusively part-time students in two-year colleges to graduate at a rate of 20.4% (number calculated by adding the three “completed at” categories). For students who are mixed-enrolled, meaning both part-time and full-time, that number moves up to 36.9%. Entirely full-time students have a whopping 54.5% completion rate after six years. Keep in mind that during these six years, 10% of two-year public college students were exclusively part-time , 67% were mixed-enrolled, and 23% were exclusively full-time. I would argue, however, that once the mixed-enrolled group is disaggregated and categorized, one would see far more students who could be considered more part-time students than full-time, and their graduation rates would slope downward as their part-time proportion increased. In fact, the American Association of Community Colleges puts the percent of part-time students in community colleges at 62%.
20%, 37%, 55%: These two-year public graduation rates clearly show a pattern. The more part-time a student is, the more difficulty that student has graduating from college. Full- and part-time status is just one factor. It is well known that minorities have lower graduation rates when compared to whites; that poor students have lower rates when compared to wealthy students; and that students with parents who do not have postsecondary degrees have lower rates than students with parents who have those degrees. In other words, within this pattern, one could easily create subcategories that are consistent. For example, within the category of exclusively part-time students, students with minority status would have lower completion rates than the average part-time rate. As another example, students whose parents lack postsecondary degrees (first-generation students) have higher dropout rates.
The same pattern can be found in four-year institutions (see Figure 3). This time the factor being analyzed is an institution’s acceptance rate. What percent of applicants do institutions accept, and how does this affect their graduation rates? According to the USDOE’s annual report entitled The Condition of Education 2017, the pattern is quite clear. If an institution accepts all students who apply, their six-year graduation rate is only 32%. If they only accept 25% or less of all the students who apply, that completion number shoots up to an astounding 88%. There is a direct linear relationship between acceptance rates and graduation. On a side note, currently about 28% of four-year colleges accept all student applicants (McFarland et al., 2017, p. xxxviii); these institutions are termed “open admissions” in the figure below.
As we know, an acceptance rate is a proxy for many factors such as SAT and ACT scores, etc., and these scores are proxies for parental income and education, probably the strongest predictors of college graduation (Cahalan et al., p. 77). In other words, the more an institution selects for higher-performing students, the better those students perform.
Moreover, if the “open admissions” column were broken down into part-time and full-time students, part-time students would have an even lower graduation rate (see Figure 14). According to the NSCRC, exclusively part-time students at four-year public colleges only graduate at a rate of 19.8%. If that same column were broken down into students who started in remedial courses versus students who did not, it would make sense that the graduation rate for remedial students would be a little lower. As with two-year colleges, the same pattern emerges for four-year public institutions when full- and part-time status is considered: 20%, 51%, 81%. (It must be noted, interestingly enough, that the difference between the public community college completion percents [20%, 37%, 55%] is almost exactly 17 percentage points each, and the difference between the three four-year college rates [20%, 51%, 81%] is almost exactly 30 percentage points each. In other words, the degree of separation between part-time, mixed, and full-time graduation rates is very consistent across institutions.)
Unfortunately, the NSCRC data above does not break down two-year public college completion data into remedial versus nonremedial students. We have to compare other data points, such as the 16% rate cited by the California community college article, to the current data we have access to from the NSCRC and elsewhere. If the 28% completion rate for remedial students reported by the CCRC was analyzed using a six-year time frame, the resulting number would probably be closer to 25%. (When tracking completion data by year, it follows the principle of diminishing returns, meaning the last two years of completion data would show the least amount of gains.)
In order to arrive at a reasonable baseline for a six-year remedial graduation rate, one could take a weighted average of the two numbers, 16% and 25%. Since the California Community Colleges System serves about 10-20% of the total number of community college students in the nation, that baseline remedial graduation rate could be calculated as 23%.
Putting it into context, then, the overall 23% remedial completion rate is lower than the two-year public college overall average of 39%, but this should be expected because we know that remedial students are more likely to be nonwhite, poor, and part-time. The main point, however, is that remedial graduation rates, even at their lowest, are very similar to graduation rates of exclusively part-time students.
To understand these numbers better in context, below are combined charts of the preceding completion data, all with a six-year time frame:
Given that two-year public remedial graduation rates are proportionately lower than the average, and they are similar to exclusively part-time rates, it is difficult to argue that remediation is the barrier that so many are claiming it is. Based on the part-time graduation rate data alone, would a researcher claim that being a part-time student is a barrier to college graduation, and thus should be eliminated?
More importantly, to be consistent, researchers should also cite the 36% graduation rate of open admission four-year public institutions and claim that the courses these students take first are barriers to be reformed. The percentage point difference between the two completion rates the CCRC cites (28% and 43% = 15 percentage points) is less than the difference between 36% and 55% (the current average six-year graduation rate of students at four-year institutions is 54.8%). Since four-year enrollments are almost twice as high (7.2 compared to 13.3 million), the low four-year graduation rates of open admissions colleges, which comprise nearly a third of all four-year institutions, could be considered more of a problem than low remedial completion rates. When taken in the context of higher education completion rates, remedial graduation rates are indeed too low, but they are in line with what is to be expected. Thus they should not be considered a barrier.
Many other researchers have come to similar conclusions based on their own research into remedial completion rates. For example, Dr. Paul Attewell et al. (2006), in an oft-cited article entitled “New Evidence on College Remediation,” make many statements supporting remediation’s successes. In this quote, Attewell et al. refer to Clifford Adelman’s prior research and findings: “Less well-known than these figures on remediation and noncompletion is Adelman’s finding that college remediation ceases to predict graduation, once a measure of secondary school academic performance and preparation is added to the model (1999, p. 75). This implies that poor high school preparation, rather than taking remedial coursework, is what reduces students’ chances of graduating from college” (p. 889).
Adelman’s conclusion is exactly the point I am attempting to make in this article. The blame for remedial students’ low completion rates should not rest on the shoulders of remedial courses themselves. Rather, the backgrounds and situations of underprepared students largely determine their completion outcomes.
As further support for this assertion, Attewell et al. go on to state that “students who took remediation at a two-year college had significantly lower graduation rates than students at the same kind of institution who did not take remedial coursework. However, after we add controls for family background and academic performance in high school, this effect is reduced to nonsignificance, in both logistic and propensity models. We interpret this as meaning that taking one or more remedial courses in a two-year college does not, in itself, lower a student’s chances of graduation. Causal factors that do reduce one’s chances of graduating include low family SES, poor high school preparation, and being Black, but not college remediation per se” (p. 905).
Other researchers find similar conclusions regarding the success of remediation. For instance, Dr. Peter Bahr of the University of Michigan finds that “skill deficient students who attain college-level English and math skill experience the various academic outcomes at rates very similar to those of college-prepared students who attain college-level competency in English and math. Thus, the results of this study demonstrate that postsecondary remediation is highly efficacious” (p. 199).
Regarding graduation as a specific outcome, an ACT research paper from 2013 found that when analyzing the “developmental students typically completed a Bachelor’s degree in six years at a rate similar to or higher than that of non-developmental students in five years” (p. ii). What this means is that since remedial students have to take more classes than nonremedial students and are more likely to be part-time, it would make sense that they take a little longer to graduate. If remediation were in fact a barrier, then remedial students would never attain similar graduation rates as nonremedial students, much less one year later.
The most important point that needs to be addressed regarding the barrier argument has to do with first-year, first-semester, nonremedial courses. Many articles are quick to point out how few students make it through their first remedial course. However, as noted, few articles note comparison data, such as the passrates of first-year, first-semester nonremedial courses. In fact, few sources cite any first-year course passrates at community colleges or universities. In a rare article, The Education Trust organization asked for and received passrates of some four-year institutions’ first-year nonremedial math courses (see Table 7).
These eight institutions’ first-year passrates average below 50%. Looking at the University of Alabama in particular, EdTrust also published an analysis of their remedial course passrates compared to their first-year nonremedial course passrates (see Table 6; Math 100 and Math 110 are nonremedial).
What this shows is that over the span of eight years at the UA, remedial course passrates were very similar to nonremedial course passrates. Both types of courses, remedial and nonremedial, are therefore barriers. If the remedial courses were the only ones to be a barrier, then the passrates of these remedial courses would have been far lower than the college-level course passrates.
In another little-known article, Zeidenberg, Jenkins, and Scott (2012) from the CCRC explore courses other than gateway math and English. They come to the conclusion that in spite of “the focus on college math and English, we found that many introductory college-level courses in other subjects also served as obstacles to completion for many students, and these latter courses posed obstacles just as great as college math and English” (p. 28). They include Biology I and II, psychology, sociology, US History I and II, computer applications, and others on a list of obstacle courses in community colleges. On a side note, this study also found that among students who graduated, there was no difference between the grades of remedial students and nonremedial students (p. 20). If remediation is a barrier, then even the remedial students who graduate should have lower GPAs than nonremedial students.
The natural response to a perceived barrier is to remove it as quickly as possible. Remediation has been classified as a barrier in many reports, and this is why it is being subject to hasty reforms. However, no researcher or policy expert is looking at any low passrates of first-year, first-semester college-level biology courses, for example, and arguing that they should be eliminated, fast-tracked, or combined with Biology II courses. Nor do we read news articles entitled, “Yet Another College-Level Course Found to be a Barrier at Community Colleges, This Time Psychology.” It is unthinkable to make these arguments about anything other than remediation. Instead of applying reforms to most initial college-level courses, researchers only seem to bemoan the low graduation rates of college students in general. For some reason, the cause of low graduation rates has been attributed almost exclusively to remedial courses, which just happen to be the first set of courses many students take at community colleges and universities.
It turns out that college itself is the barrier. As national retention rates demonstrate, just below 70% of first-time students who enroll at four-year public institutions in a fall semester return the following fall semester, with part-time students near 40%. At two-year public institutions, fall-to-fall retention rates are around 47%, with part-time students lower than 40% (Snapshot Report No. 18, p. 6). In fact, college dropout rates are steepest from from the first semester to the second semester. Researchers who simply look at remedial course outcomes—even just the students at the cutoffs of these courses—and then conclude that these courses are the main barriers, are making a classic error in data analysis, confusing causation with correlation.
If remediation is a barrier, then any course that any student takes is also a barrier, especially one taken in the first semester of college. And the more college courses students take, the higher the chances they have at dropping out. Of course, at-risk students carry more potential barriers with them than other students when they begin college. That is why limiting or removing access to fundamental prerequisite courses at four-year and two-year colleges is not the answer. In fact, the opposite should occur. Substantial resources should be infused into remedial courses in order to improve them, while linking them with an entire network of well-supported areas in colleges, such as admissions, counseling, tutoring, student success, financial aid, etc.
This is what separates “developmental education” from “remediation.” Even though most researchers use the terms interchangeably, actual developmental education, as defined by experts in the field, has always been meant to include more than remedial courses alone. It should be a system of support, well-funded and thoughtful, which includes remediation, especially prerequisite, traditional remediation for those students who are underprepared.
An excellent example of developmental education in practice already exists. It is called the Accelerated Study in Associate Programs (ASAP), and it is housed at the City University of New York. They invest thousands of dollars per student per year, form a network of thoughtful support for at-risk students, more than double their graduation rates in three years, and still capitalize on a high return on investment. And they use traditional remediation courses at the beginning of the program. Clearly these courses are not holding students back.
We need to realize that mislabeling an entire set of tiered, sequential courses as a barrier is not helpful. What we need to do instead is thoughtfully support citizens in search of a higher education instead of cutting their access and opportunity to college.
Adelman, C. (1999). Answers in the Tool Box: Academic Intensity, Attendance Patterns, and Bachelor’s Degree Attainment. Washington, DC: U.S. Department of Education. Retrieved from http://files.eric.ed.gov/fulltext/ED431363.pdf
Adelman, C. (2004). Principal Indicators of Student Academic Histories in Post-Secondary Education, 1972–2000. Washington, DC: U.S. Department of Education, Institute of Education Sciences. Retrieved from https://www2.ed.gov/rschstat/research/pubs/prinindicat/prinindicat.pdf
Attewell, P., Lavin, D., Domina, T., & Levey, T. (2006). New evidence on college remediation. Journal of Higher Education, 77(5), 886–924. Retrieved from http://knowledgecenter.completionbydesign.org/sites/default/files/16%20Attewell%20JHE%20final%202006.pdf
Bahr, P. R. (2010). Revisiting the efficacy of postsecondary remediation: The moderating effects of depth/breadth of deficiency. Research in Higher Education Journal, 33(2), 177–205. Retrieved from http://debdavis.pbworks.com/f/revisiting+postsecondary+remediation.pdf
Cahalan, M., Perna, L. W., Yamashita, M., Ruiz, R., & Franklin, K. (2017). Indicators of higher education equity in the United States: 2017 trend report. Washington, DC: Pell Institute for the Study of Higher Education, Council for Education
Opportunity (COE) and Alliance for Higher Education and Democracy (AHEAD) of the University of Pennsylvania. Retrieved from http://pellinstitute.org/downloads/publications-Indicators_of_Higher_Education_Equity_in_the_US_2017_Historical_Trend_Report.pdf
Gordon, L. (2016, November 9). Remedial courses are ‘barriers’ for many community college students, report says. Edsource. Retrieved from https://edsource.org/2016/remedial-courses-are-barriers-for-many-community-college-students-report-says/572483
Jaggars, S. S., & Stacey, G. W. (2014). What we know about developmental education outcomes. New York, NY: Columbia University, Teachers College, Community College Research Center. Retrieved from http://ccrc.tc.columbia.edu/media/k2/attachments/what-we-know-about-developmental-education-outcomes.pdf
McFarland, J., Hussar, B., de Brey, C., Snyder, T., Wang, X., Wilkinson-Flicker, S., Gebrekristos, S., Zhang, J., Rathbun, A., Barmer, A., Bullock Mann, F., and Hinz, S. (2017). The condition of education 2017 (NCES 2017- 144). U.S. Department of Education. Washington, DC: National Center for Education Statistics. Retrieved from https://nces.ed.gov/pubs2017/2017144.pdf
Noble, J., & Sawyer, R. (2013). A study of the effectiveness of developmental courses for improving success in college. Iowa City, IA: ACT Research Report Series. Retrieved from https://www.act.org/research/researchers/reports/pdf/ACT_RR2013-1.pdf
Shapiro, D., Dundar, A., Wakhungu, P.K., Yuan, X., Nathan, A. & Hwang, Y. (2016, November). Completing College: A National View of Student Attainment Rates – Fall 2010 Cohort (Signature Report No. 12). Herndon, VA: National Student Clearinghouse Research Center. Retrieved from https://nscresearchcenter.org/wp-content/uploads/SignatureReport12.pdf
Snapshot report no. 18: Persistence-retention. (2015). Herndon, VA: National Student Clearinghouse Research Center. Retrieved from https://nscresearchcenter.org/wp-content/uploads/SnapshotReport18-PersistenceRetention.pdf
Yeado, J., Haycock, K., Johnstone, R., & Chaplot, P. (2014). Higher education practice guide: Learning From High-Performing and Fast-Gaining Institutions. Edtrust.org. Washington D.C: The Education Trust. Retrieved from http://edtrust.org/wp-content/uploads/2013/10/PracticeGuide1.pdf
Zeidenberg, M., Jenkins, D., & Scott, M. A. (2012). Not just math and English: Courses that pose obstacles to community college completion (CCRC Working Paper No. 52). New York, NY: Columbia University, Teachers College, Community College Research Center. Retrieved from http://ccrc.tc.columbia.edu/media/k2/attachments/not-just-math-and-english.pdf