Adamczk, B., Reffeor, W., Jack, H., “Math Literacy and Proficiency in Engineering Students”, ASEE Annual Meeting, Montreal, Quebec, June 2002.

Abstract

After years of anecdotal evidence of math deficiencies in engineering students a study was conducted to assess the math proficiency of sophomore and junior students at Grand Valley State University (GVSU). The study confirmed some of the expected outcomes and revealed some unexpected surprises. The analysis of the test scores is presented correlating the scores with the previous math classes and ACT math scores. The results of the math assessment prompted the development at GVSU of short and long term strategies to remedy the problems. These strategies and their impact are discussed in detail.

Introduction

Grand Valley State University (GVSU) offers ABET accredited programs in Mechanical, Electrical, Computer and Manufacturing Engineering. The program includes a secondary admission process to permit entry into the fifth semester and mandatory co-operative education employment. Any student who has been admitted to the university is permitted to take courses in the first four semesters of the program, providing he/she has the appropriate prerequisites. The student population is a combination of transfers from other schools, and first time university students.

Students who transfer to GVSU often bring a majority of the required mathematics credits with them. First time students entering engineering are given placement tests to help select their first mathematics course. The placement test, developed by the Mathematical Association of America, is provided by the Math Department, but is administered and graded by the engineering faculty during freshman student advising. Students are forewarned about the test, and they are permitted to use calculators when taking it. The score on the test is used to place students in the courses listed below in sequence. A calculus ready engineering student will normally start in MTH 201 (Calculus I). Students who are not calculus ready are placed in MTH 110 (Algebra), 122 (College Algebra) or 123 (Trigonometry). These students are much less likely to complete the engineering program.

Calculus courses:

MTH 201 Calculus and Analytic Geometry I

MTH 202 Calculus and Analytic Geometry II

MTH 203 Calculus and Analytic Geometry III

MTH 302 Linear Algebra and Differential Equations

The primary statistics used for admission to the university are the high school GPA and the ACT score. The secondary admission process is primarily based upon the students’ academic performance in the first four semesters of the program. The average ACT scores and math placement of students for the students admitted to the program in 2000 and 2001 are shown in Table 1.

Table 1: Average ACT Scores and Math Placement

Year	Math ACT	Overall ACT	MTH 110 Algebra (or lower)	Math 122/123 College Algebra/Trig.	MTH 201 Calculus I
2000	27.3	25.3	16%	5%	79%
2001	28.1	26.7	8%	16%	76%

Note: The statistics do not involve transfer students from other institutions

The secondary admission process guarantees the quality of the program, and the result is a very low attrition rate in the upper years. However, the attrition rate before the secondary admission process is very high. The pre-admission attrition rate is increased by inadequate mathematical skills. For students not placed into MTH 201, at least one year is added to their program. Even when students have been placed into MTH 201 they often have problems with basic mathematical operations. This often decreases their performance in other courses. And, even after the secondary admission process, many of these problems persist. The remainder of this paper will discuss the path we have been following to identify and solve these problems.

History

As seems to be the trend in many universities [3][4], in the early 1990s, faculty at GVSU began to notice that students were having difficulty performing even the most basic mathematical calculations. In particular, third semester students in EGR 209 (Statics and Mechanics of Materials) were unable to perform many basic mathematical calculations. A math review session, up to three lectures in length, was added to the course. Although this impacted the available lecture time, it reduced problem based delays in following lectures. In general the material reviewed included basic algebra, trigonometry and vectors.

Math problems were also observed in the fifth semester engineering class, EGR 345 (Dynamic Systems Modelling and Control). The students were having problems solving basic first and second order differential equations. To assess and remedy the problem the solution of these equations was reviewed and the students were tested on the methods. In total, the students were tested three different times, each with prior warning. Each of the tests included a second order differential

equation. On the first test the equation had two real roots, the second test had two identical real roots, and the third test had complex roots. The test scores for the third test are shown in Table 2. The first column shows the test scores out of 10. Typically students receiving less than 6/10 were unable to successfully complete the homogeneous solution. A student that completed the problem but made a minor error typically received 8/10. The second column shows the grade the student received in the GVSU differential equations course. In all cases but one, the students who had received transfer credit for the differential equations course were unable to solve differential equations. The third columns shows the students who had transferred from other institutions. In general these students were distributed throughout the course. The differences between the second and third column indicate that transfer students are not a specific source of problems, but transferred courses in differential equations are.

It was also useful to note the correlation between the grades in 302 and test grades as shown in Table 3. This shows in general that the MTH 302 score is a weak indicator of student knowledge of the subject.

Broader Testing

In the fall of 2001 math assessment exams were given to all of the fifth semester (junior) engineering students in EGR 314 - Circuit Analysis II (27 students) and EGR 345 - Dynamic Systems Modeling and Control (42 students), and the third semester (sophomore) students in EGR 209 (60 students). In the junior classes the test was presented without prior warning, while the sophomore class was given three lectures of math review and formal warning. A test was also given to the fourth semester sophomore class in EGR 214 - Circuit Analysis I (71 students) in the winter of 2002. Appendix A contains a combined set of questions from all of the math tests. The test questions vary between classes to reflect the theoretical requirements of the courses. (Note: the question used for the tests were not identical because the tests were not originally designed for a scientific study.) The tests were administered so that students had ample time, but calculators were not permitted.

The results for the tests are shown in Appendix B, Table B.1. The columns show the percentage of students able to successfully complete the calculations. In this case the electrical and computer engineering juniors in EGR 314 and the mechanical and manufacturing juniors in EGR 345 had similar levels of performance. In particular consider the algebra questions. In this case the students were not forewarned, but the simplicity of these questions would suggest much higher scores. Also surprising is the poor performance on the calculus questions after completing the four course calculus sequence in their previous semesters. For example consider question 6 which showed that half of the students were unable to apply the product rule to solve an integral. We speculate that the poor performance is the result of the multi-step nature of the problem. Other questions, such as 7c, are topics that instructors assume are well understood.

The combined test scores are shown in Table 4. It shows that the test results in EGR 209 were higher, as would be expected with a review session and prior warning. The contrast in the differential increase between the algebra and calculus scores is interesting. It suggests that the students had knowledge of, but were not proficient with algebra, and the review was able to increase the proficiency by 28% over the junior students. But, the trivial increase in the calculus scores suggests that there are knowledge gaps in the material, and review only increased proficiency 2.5% over the juniors.

Table 4: Overall scores by subject area

	EGR 209	EGR 214	EGR 314	EGR 345
algebra	91%	70%	66%	60%
calculus	74%	68%	76%	67%

In summary these results indicate that review sessions are required to increase proficiency in topics that most educators assume are fundamental. It also indicated that there are systematic calculus problems that must also be addressed. Without preparation, we can expect error rates of about 30% on the most trivial calculation steps.

Relating the Results to Students Backgrounds

The students in one of the junior level classes were examined in detail in an attempt to identify sources and/or indicators of math problems. A selected set of data is shown in Appendix B, Table B.2. It is ordered based on the test scores from Table 2. The second column of the table shows the ACT math scores, out of a possible score of 36. In the cases where the scores are missing the students started elsewhere and transferred to GVSU. The ACT math score is highly variable with respect to the test score, although there is a general correlation. In particular there are a number of higher ACT scores in the 30’s that have very poor math scores. This variability suggests there is another factor not measured. The only major variation that seems to explain the difference is that the ACT test permits the use of calculators, while we did not. This suggests the conclusion that some students are using the calculators as a ‘crutch’ and don’t actually understand the fundamental material.

The column labelled ‘first math’ shows the entry level math course the student took at GVSU or elsewhere. The students who were calculus ready, and started in MTH 201, were more likely to score well on the test. Of the 13 students who started before MTH 201, 2 scored above 70%, while 12 of 31 calculus ready students scored above 15. The last four columns list their math scores in the calculus course sequence. Grades are not reported for students who took the courses from other institutions. The students who received grades of ‘A’ generally scored better on the math test. But, there are some notable anomalies and students who received grades of ‘B’ or ‘C’ are scattered in performance. Surprisingly the lack of correlation indicates that the students math proficiency cannot be clearly predicted by their grades in their math courses.

A Review Course

Students who performed poorly on math courses were offered an optional review course in the fall of 2001. The course was run for three hours on four consecutive Saturdays. The sessions allowed the students to solve basic engineering problems from an assigned review book [1]. The topics covered are listed in Table 5. These topics were chosen to review basic topics that should be common knowledge to all engineering students.

Overall the review course benefited students, as shown in Table 6. In the table, 7 sophomore students who attended the Saturday course were tracked from their test scores EGR 209 to EGR 214. The first column shows the students raw score, while the second column shows the score compared to the class average in EGR 209. The third and fourth columns are for their performance in EGR 214, after they attended the review course. The difference shows the students improvement relative to their peers. Overall four students had a major improvement, and only one student worsened substantially. Numerically, there was an increase of 14%.

 

Table 6: Student performance before and after the review course

EGR 209 score	% of class 209 average	EGR 214 score	% of class 214 average	difference
68%	91%	87%	96%	-4%
43	57	50	72	26
72	96	90	130	35
68	91	80	116	27
68	91	50	72	-21
68	91	90	130	43
57	76	50	72	-5

The review course focussed on drill-and-skill problem solving, and as shown by the improved scores this was a fruitful approach.

Conclusion

We are now at the point where we have identified problems and are beginning to solve them. As shown before they don’t originate from a single source. Many of the topics, such as algebra, are the responsibility of the K-12 schools, while others, such as calculus, are clearly the domain of higher education institutions. The K-12 schools do not appear to be using enough drill-and-skill to foster problem solving proficiency[2]. The college level system doesn’t seem to be imparting proficiency or knowledge in some cases. We speculate that this is based on the shift to concept based curriculums, supported by calculator and software usage.

Bearing the brunt of this problem are the students who are responsible for the knowledge, but have been inadequately prepared. If we consider students’ mathematical knowledge as a scaffold, each layer is poorly constructed, and each new layer is on progressively weaker footings. At each point the student is unable to fully articulate new topics because they are continually rebuilding their knowledge of the basics. This also hinders instructors who are continually trying to shore up the mathematical weaknesses. This continuous revisitiation of previous topics eventually frustrates students and faculty alike.

In summary what we learned was:

 

- Assessment exams should not permit calculator usage

- Math courses should focus more on drill and skill to develop proficiency

- There is a high level of variability between math courses and institutions

- We cannot assume that any of the fundamentals are well understood

- Reviewing math topics may be required for fundamentals

- Multiple step problems pose a high level of difficulty for students

References

[1] Frank Ayres, Jr., Philip A. Schmidt, College Mathematics; Schaum's Outlines, Second Edition, McGraw-Hill, 1992

[2] Mike Robinson, M. Sami Fadali, How Do Secondary Science Texts Cover Mathematics and Engineering Principles and Design?, ASEE Conference Proceedings, 2001.

[3] Wilson, R., “A Decade of Teaching ‘Reform Calculus’ Has Been a Disaster, Critics Charge”, The Chronicle of Higher Education, A12-13, Feb., 7, 1997.

[4] Wu, H., “The Mathematics Education Reform: Why You Should be Concerned and What You Can Do”, The American Mathematical Monthly, 104(10), pgs. 946-954, Dec., 1997.

Appendix A - Combined Test Questions

1. Simplify the following expressions.

2. Manipulate the following equations to solve for ‘x’.

3. Express in the common denominator form:

4. Solve for a:

5. Express the following equations in matrix form:

6. Find the following derivatives.

7. Find the following integrals.

8. Evaluate dy/dx.

9. Determine the second derivative of

10. Evaluate

Appendix B - Test Results

Table B.1: Detailed test results by class and topic

Topic	Question	EGR 209	EGR 214	EGR 314	EGR 345
algebra	1 a	84%		78%	60%
	1 b	99%		34%	25%
	1 c	93%		74%	72%
	1 d			66%	68%
	1 e			69%	78%
	1 f	99%
	1 g	98%
	2 a	93%		83%	70%
	2 b	69%		57%	44%
	3		70%
	4		93%
	5		46%
calculus	6 a	96%		83%	89%
	6 b	87%		62%	56%
	6 c	80%		59%	52%
	6 d	83%		74%	74%
	6 e	18%
	7 a	95%		91%	89%
	7 b	91%		76%	51%
	7 c	66%		69%	45%
	7 d	84%		93%	78%
	7 e	61%
	7 f	56%
	8 a		97%
	8 b		55%
	8 c		31%
	8 d		56%
	9		97%
	10 a		68%
	10 b		75%