Journey to Interleaved Practice #3

In my first post in this series I shared my thoughts on my motivation for the design the observational study, noting that discrimination was a key idea I wanted to explore. In my second post in this series I shared some of the tools and my thought process in designing the structure of the interleaved homework assignments. In my final post on my journey (for the season, anyway), I will share some preliminary results, some student solutions that I found interesting, and my overall thoughts on what I learned.

Preliminary Results

First, here are the overall trends in the assessments from this term.


Each solid line represents one of the 14 students who were involved in the observational study. The dashed black line represents the average progress of the class. A few things should be immediately apparent:

  1. The black line shows a general decline over the semester of about 20% if one observes Quiz #1 first and the Final Exam last. However, if one were to remove the quizzes, one would see a decrease of 15% from Test #1 to Test #2, followed by a slight increase of about 5% from Test #2 to the Final Exam. More discussion on this below.
  2. What the heck happened to that poor blue student? It might be that H found the interleaved structure of the course and homework overwhelming and needed more time for comprehension compared to the other students. Is it possible that students with special considerations benefit more from the structure of a blocked approach? I haven’t read much on this, but please feel free to share some research if you know about it.
  3. Aside from a few students who remained close to the top for the assessments, many students saw a drastic decrease around Test #2. Why is this? Test #2 contained 86 points dedicated to all the various integration techniques (substitution, integration by parts, strategies for trigonometric integrals, trigonometric substitution, partial fractions) and I told my students to do whatever questions they wanted to in order to obtain 50 marks. Perhaps this choice was too much, and a more structured test would have been better-suited.

If there are other items that are particularly noticeable, let me know and I will reflect a bit more on why that might be the case.

I also compared the scores of the 14 students on Test #1, Test #2 and the final exam from differential calculus to integral calculus. Since Test #2 was so varied from the structure of differential calculus, I decided to exclude it here (although there was a 10% decrease). Test #1 saw a change in scores of about 10% and the final exam also showed a slight increase in score of about 2%.test1test2

Student Solutions

A estimates the area under the upper half-circle using a parabola.
B shares how we know the integral is bounded above by 1 using properties of odd functions. 
M shows the correct (and challenging) solution to two difficult integrations on Test #1.
What I find odd about P’s solution is that she gets the substitution in part (a), yet tries to use the power of -1 in part (b). This was a common misconception in the course that persisted all semester. 
Trigonometric integrals offer interesting flexibility in answers. Here, I would have tried a substitution, but A decided to make everything cosines and use cosine reduction.
Here, a u-substitution would work, but N decides to use trigonometric substitution.


First and foremost, while I did select an interleaved approach due to the hopes that it would make integral calculus a bit easier in the long run by allowing students to discriminate between integral techniques, I also noticed that students’ mindsets changed a bit this semester. In differential calculus, where they might not venture an answer, in integral calculus they would try substitution or integration by parts, even if it led them down a dangerous path. There was a difference in both effort and execution. They persisted and often came up with insightful solutions. It was also true that there was less cramming for tests and the exam. In fact, N came up to me and said “I realized that I had to study some material less because I knew how to tackle these problems. It’s kinda fun.” It would be interesting to follow-up with them over the summer months to see how much of this knowledge they retained.

From my perspective, I know that any fluctuations in grades are highly likely due to random chance factors, and not necessarily due to the interleaved practice. This said, it was an interesting first-go at something this big and I definitely want to try it again. The main difficulties I had were:

  1. Time. It took a lot of time to work through the homework solutions in class. Due to the time I lost, I had to teach differential equations in the lab portion of the course, and lost time discussing some aspects of power series. I’m not sure I would have necessarily changed this, as many students appreciated the extra time spent on solving questions and being able to ask specific questions.
  2.  How do I measure whether or not the interleaved practice actually helped? I’m not sure that I effectively can do this based on the way the study is designed, but here is a thought. When a student tackles a question, either they use the correct technique or they don’t. What if I looked at the proportion of times a correct technique was used on Test #2 and compare it to the proportion of times a correct technique was used on the final exam? Maybe this would be helpful.

Unmasking Logarithms

I had such an interesting conversation in my pre-service math class the other day. We were solving the equation


My goal here was to get them thinking about how they could us the power laws to help them. We worked our way down to


And someone offered the suggestions that the 3^8 and the 3^4 x 3^4 were the same, so all we really needed to determine was

CodeCogsEqn (2).gif

and we eventually settled down on x = 2, since 4^2 = 16. Then I did something weird. I told them to pull out their calculator and evaluate

CodeCogsEqn (3).gif

to which they found the answer to be 2. Now I had them intrigued. How were logarithms connected to this question?!

They knew that logarithms were a pre-calculus operation, but hadn’t made that connection between logarithms and exponentiation. It is likely that logarithms had been taught as a series of rules to follow, without a real explicit connection to how they are the inverse operation of exponentiation – or, more importantly (in my opinion) how they solve one piece of the “triple puzzle.”  You see, the process of exponentiation involves three values: the base (a), the power (p), and the evaluation (b).

CodeCogsEqn (4).gif

We could cover any one of these numbers up, so we have three different but related problems. (1) We could cover up b; this problem can be solved by the process of exponentiation.


(2) We could cover up a; this problem can be solved by applying a radical.


(3) We could cover up p; this problem can be solved by applying a logarithm.


I am not convinced that students get enough time exploring  and developing their sense of logarithms, so I suggest utilising a structure that was brought up by David Butler today (the triangle typically used in science courses to remember arrangements of formulas). I do think in our case, the structure of the triangle works a bit better than it does for science formulas. Here, the triangle works nicely for a^p = b:


What if we fill in two values:




Is it possible that we can create meaning about logarithms by using these diagrams to introduce the three similar, but related, problems? I think so. We know 2^3 = 8, so how might we reason through this?


Well, we know ? must be close to 3 since if ? = 3, we would get 8. We also know ? must be larger than 2 since 2^2 = 4. Hmm… here we might begin to introduce the clunky notation of logarithms. Perhaps log(7)/log(2), or log 2(7). Aha! 2.807 seems reasonable based on what we have thought about. And we can now flush out the problem of non-integer powers.

Anyway, I don’t think we will ever be able to get rid of the unfortunate notational issues with logarithms, but I do think we can do better making the connections back to exponentiation. Maybe there is some space in the progression of learning about exponentiation for triforce notation? As always, I welcome your thoughts.

Journey to Interleaved Practice #2

In my last post, I gave some background to the study that I am undergoing with my calculus students this term. In this post, I want to share some of the tools and methods I used to make the path towards interleaving clearer to me.

I have been a fan of interleaved practice for some time since it is well-known in the scientific community to be a successful strategy for learning (here I am thinking about learning as a flexible and long-term change in long-term memory that can be measured through test performance). However, when thinking about how to successfully implement interleaved practice it feels like a very daunting task and there are a lot more questions compared to answers:

How many questions do I assign at each step?
How do I best mix-up all the questions?
Should some topics be more weighted compared to others?
In what order to I teach the topics? Should I also interleave the way I teach the topics?

So what I did was draw some inspiration from a Slack work-group where Yana‘s husband Fabian (congrats on the recent wedding!) put together an Excel worksheet that gave a potential teaching and quizzing structure using an interleaved approach. If you open this link, you can see space to enter the topics, as well as the number of classes you have, and finally the number of questions you want per quiz. Hitting the “Do Quiz” button will create two lists: one that suggests topics to teach during any particular class, and one that suggests the topics for each quiz (which I assume happens at the beginning or end of each class).

I took this basic structure and decided to create a list of potential topics for my integral calculus course. I divided this list into six “strands” each with a certain number of “lessons” (note: I am not done finalizing this list yet – it is a work in progress). Basically I sat down, went through each chapter of the textbook and made a map of how the topics were interconnected. For example, the Sequences and Series section of the textbook discussed geometric series. Well, I could easily do this in Strand One so that students have an introduction to sigma notation before working with sigma notation with approximations. Then I could circle back to sigma notation later in Strand Six when working with Taylor Series, effectively spacing out our work with sigma notation throughout the semester. Each placement of a lesson within a strand was a calculated choice to try to space out the important ideas as best I could.

Now that I had a list of topics, I could input this information into Fabian’s worksheet and get an idea of how to interleave topics. I decided that I had already interleaved teaching topics as best I could, so I ignored the top output. I chose 4 questions per quiz and focused my attention on the bottom output. Using the output at the bottom as a model, I created a new page that listed the four questions I wanted to include on each quiz. See Sheet 3 of this workbook for that page (again a work in progress).

My final decision was not to use quizzes, but homework assignments instead. That is, at the end of each lesson, I give a PDF handout like this one to each student that is due at the beginning of the next class. This particular PDF came after the lesson on the Fundamental Theorem of Calculus Part II (FTC II). Notice that there are questions about the FTC II, but there are also questions on the topics of geometric series and the definition of the definite integral as well (the limit of the Riemann sum question).

To ensure that students complete each 4-question homework assignment to the best of their ability, I check for completion only at the beginning of class. We then take the questions up as a class – focusing on the “hard” questions that students are having trouble with. So far things have been going very well. The first test is coming up next week, and I will definitely try to blog about any interesting information I gather from looking at their responses.

Journey to Interleaved Practice #1

This semester I decided to create a study on interleaved practice with my second-semester calculus class. By no means is the study empirical in nature – I am not using any controls, and haven’t thought much about confounding variables. The study is more observational in nature, with the goal of collecting student solutions to analyze how students are answering specific questions.

The idea came about through an email discussion with Yana Weinstein of the Learning Scientists (and University of Massachusetts, Lowell), and Doug Rohrer of the University of South Florida. I had been interested in using some of the interleaved practice tools that Yana had helped develop in our Slack team, and she thought it would be nice to touch-base with Doug, as he thinks a lot about how interleaved practice affects students’ learning in mathematics.

There were two specific papers that I remembered reading, this being one of them. I thought the discussion on discrimination rather appealing, and something that I tended to see each semester. Roughly speaking, we teach mathematics in a particular way, scaffolding from one idea to the next, with practice questions always coming from specific chapters. As students practice, they always know the strategy that they need to use in order to solve the question (ie. they think “the questions are at the end of the lesson on the Pythagorean Theorem, so I probably have to use the Pythagorean Theorem to answer the question”). They don’t, however, get much practice mixing the different strategies that they learn. Unfortunately, this means when they come to a summative test, extra effort has to be initially put in to determine what strategy to use to solve a question.

My main goal is to do a bit of observational research around discrimination on summative tests. I have developed a schedule of interleaved homework and interleaved lessons for my integral calculus class, and we are currently off to the races. When I check back in next time, I will share some of the tools I am using to create the interleaved homework.

A Calculus-Themed Amazing Race

On Monday this week, I ran an Amazing Race event on my campus where my calculus students moved from location to location solving review questions and racing against the clock for extra credit.

The hardest part for me was thinking through the original logistics of the event. Luckily with the help of our campus recreation coordinator, Jo Ann, we developed a preliminary schedule with colour groups. The premise was the we would divide students into groups of three, assign a colour group, then have a series of four clues at various locations around campus. At some locations, the clues would be hung on the wall, and at other locations the group would have to interact with someone (eg. our receptionist) to receive their next question.

Schedule of colour groups and locations of clues.

Now that the schedule was created, it was up to me to design questions that gave an appropriate clue letting the team know where the next question was. This was a bit tricky, but after a few hours of hard work, I had a series of questions that gave clues:

  1. Room PE 10, clue: p'(e) = 10
  2. Room PE C14, clue: C = 14
  3. Room PL 117, clue L = 117
  4. Room PC 133, clue 133
  5. Room PC 146, clue C = 146
  6. Room PC 108, clue C = 108

So you can see for the most part, the parameter C was involved and students were somehow solving for C to receive their hint as to what room to go to. My favourite was designing a slope question in which when x = e was substituted, the answer was 10. This not only gave the number of the room, but the building too (really proud of that one)!

Students solved a limit question for a parameter L to get their clue to the next location PL 117 (our library).

When the day finally arrived, I got to school early to distribute the envelopes containing the clues in their various locations. Each envelope was colour coordinated, and contained a coloured paper. This way students & staff helping would know which was the correct envelope.

Envelopes hung at the Student Union Office PE C14.
Colour coded paper containing the question, and a marking on the envelope telling students which clue number and what colour.

Overall, I believe the students had quite a bit of fun with the activity (I heard rumor that they were telling other students about how much fun this math class was as they ran around campus). Also, the questions were challenging, but not overly taxing, which allowed a bit of group work along the way (not one student could brute-force through all of the questions). Most teams completed in about 50-60 minutes, and I ended up giving out extra credit to all who participated (tiered so that the “winners” received more). I will absolutely use this activity again – and may put in some extra locations and questions to make it a bit longer. Thank you to all the staff who helped out!

Students working together on the first clue.

The PPT Template I used for locations where I hung envelopes on the door.

My List of Questions for each colour group.

The introductory PPT Template stating ground-rules and prizes for winners.
(of importance here is that I required full-solutions and all group members to be back in order to get their final question)

What I Learnt from #rEDWash

Well, it has been officially one week since I took flight to D.C. to attend and speak at researchED Washington. And what a wild ride it was. I returned to class this week on a high – completely abuzz from meeting with people who were willing to listen about what matters in education.

I decided that I wanted to write a post about researchED, but wanted it to be more reflective in nature. So here are a few lessons that I learnt from researchED Washington:

Setting up routines is important.

I thought David Didau hit on a few important ideas in his talk Poor Proxies for Learning. One that stuck out to me was the idea that anything that occupies working memory resources reduces our ability to think, and that we need to think about something in order to learn it. I was wondering how do I lessen or eliminate unnecessary distractions in my own teaching? To me, this feels tied to Tom Bennett’s discussion about the three Rs of classroom management – the first R being routines.

Should I be more mindful of the routines that I am setting up in my college classrooms? How do I feel about technology? I am mindful to tell students that they need to think deeply about something to be able to learn it, and that cell phone use in class often leads to decreased learning capability – however, I don’t generally enforce non-use of cell phones in my classes. Would it be weird to get into a routine of no cellphones in class? This would help eliminate some necessary distractions and allow students to focus more on what matters – thinking about the mathematics we are working on. I need to spend more time on this as I move forward in the future.

Teachers don’t seem to be ready.

Maybe all of us that got together in Washington are those pioneers harvesting good things to come. But I look back, thinking to myself, “Geeze, where are all the teachers?” I like that the event had a mixture of policy-makers and experts in different fields that were interested in the future of education, but it still felt lacking in teacher attendance. Maybe I am setting my hopes too high, and that I should buy into the “Build it and they will come” mentality. Or is it that teachers in their current state, see little utility in an even like researchED? The NCTM regionals in Phoenix and Philadelphia occurred at the same time as researchED Washington, and their teacher attendance and ticket prices were much higher (researchED is a non-profit, thus ticket prices can remain low). Either way, I left the event thinking about what might make the event more useful for teachers. How do we truly make researchED a space in which practitioners can seize control of their own professional development, rather than it becoming an echo-chamber of like-minded individuals?

Good teaching boils down to more than jumping on the latest bandwagon.

One thing I really enjoyed about Cassy Turner’s talk was that she gave some indication of the power of the Singaporean mathematical ideas. The big take-away for me was that the bar-modelling, while a very strong visual in of itself, is not the end goal. The bar model is meant to introduce mathematical ideas in a visual way to allow for pattern recognition – movement to the abstract is always the end-goal. That is, there is a clear destination: building fluency with numbers or with algebra.

A potential problem arises when we adapt a particular soup de jour, without critically thinking about how it is connected to present or future mathematical ideas. Eric Kalenze discussed this through the lens of over-correction: an adaptation to a current problem without thoughtful analysis of what the problem entails. I feel like there are also connections to my talk on teacher training programs needing to contain horizon content knowledge, or knowledge of connections between mathematical ideas, and how one mathematical idea progresses to the next.

One of my highlights last weekend was responding to the question “So what do you make of Dan Meyer?” In my personal journey as a mathematics teacher, I find myself returning back, time and again, to the practices of Dan Meyer. So my response was that there is something interesting happening there – I can’t quite put my finger on what it is that I like, but I think it holds promise. Over the next month or so, I am going to spend some time reading up on some of Dan’s educational interests and trying to formalize some of the aspects of Dan’s teaching practices that I particularly enjoy, and maybe even some of the practices that I don’t. Be on the lookout for that over the holidays – and many thanks to Dan for supplying a few articles to start my journey.

It is intimidating to party with the cool kids.

While I like to pretend that I am as cool as a cucumber – I actually care a fair bit about how others perceive me. On bad days, it can be particularly challenging, as it becomes too easy to get caught up in the exhausting hustle of “How do others perceive me?” Perhaps the stress of preparing and presenting at the conference got to me, but I walked away thinking that I had let my audience down. One of my audience members left before I was able to finish. Was it because we started a bit late? Did that person really need a coffee? These things nag at me, and make me feel a bit disheartened. Maybe I didn’t deserve my spot alongside the other speakers? If I ever get to hang out with the cool kids again, one thing that I can guarantee is that I will make sure I am ready to play.

For those of you interested in the slides from my presentation, you can find them here: bridging-mathematics-mathematics-education.

When is “Doing” Enough?

So I have come to that subtle realization that just because a student can algebraically work through an example, doesn’t necessarily mean that they possess the concepts I would like them to know. That is, I am slowly realizing why so many educators are concerned that students can not only show their work, but also explain their work; and why “conceptual understanding” and “big ideas” have moved to the forefront of mathematics education. This blog is a walk through my thoughts for limits in first-year Calculus and is somewhat based around the ideas that Barry Garelick & Katharine Beals discussed in the Atlantic  last year.

My Experiment and Hypothesis

In my first-year calculus class, I wanted to get a sense of whether or not my students understood conceptually what a limit was. We had discussed them somewhat informally using the definition that “as x gets close to a, f(x) get arbitrarily close to L” and not in an epsilon-delta kind of way. They came up naturally as a way to help us answer the question “How do we define 2^pi?” in which we observed two one-sided limits (over the rational numbers) converging to a decimal number somewhere around 8.82. That is, their first work with limits came as observing one-sided limits in chart form. From here, I connected limits to graphical representations using Desmos and sliders, and came up with a few introductory rules (such as when we are allowed to use the substitution rule). After this, we worked on more of the algebraic rules: factoring, conjugation, vertical & horizontal asymptotes.

By the end of all of this, I was curious as to how they were thinking about and visualizing limits. I came across this paper, borrowed a few of their questions, and created a Desmos lab to try to help me determine this. I hypothesized that my students would be very good at the algebraic portion – that is, given a limit, I figured they would be great at working through this kind of question and getting to a final answer. However, I didn’t think that they would be as keen conceptually – that is, I thought they would have some major misconceptions about how to think about a limit (for instance, they would think that a limit is an approximation). So, let’s dive in and compare how their answers on the first test (algebraic) compared/relate to their answers from the Desmos limit lab activity (conceptual).

Test Results

The test results weren’t as good as I had originally hoped. This may be to a few factors – not enough studying or poor study habits on behalf of the students, testing being a stressful situation for the students, not assigning or covering enough questions on my behalf, the questions being too challenging on my behalf. Overall, the limit question was out of ten marks and had five parts. The mean and median mark was 4.5/10, with the highest mark being 9/10 and lowest being 0.5/10. The first three parts were standard, (a) being a conjugation [2 marks], (b) being an analysis of a horizontal asymptote [3 marks], and (c) involving factoring and analysis of a vertical asymptote [3 marks]. The last two questions were particularly challenging, with students requiring some knowledge of the asymptotes of arctan(x) and ln(x) to be successful [1 mark each]. The first three parts were definitely done better than all five parts combined; however, I did not parse these results.

So, at best, I can say that algebraic simplification of limits was average from a whole-class perspective. With that said, it might be interesting to parse down to individual student responses and analyse these to make some comparisons/contrasts. I have picked out three students that did exceptionally well with algebraic simplification of limits (some responses shown below). Let’s label these students as Student A, Student B and Student C. I am curious if exceptional knowledge of the procedure for solving limits is equivalent to a excellent conceptual knowledge of limits.


Student responses to the limit questions on the test.

Lab Activity Responses

Question: How do you feel about the following statement “A limit is a number past which a function cannot go.”

Student A: Depending on the function, the limit is a number the function approaches at a certain value of x.
Student B: True.
Student C: A function can approach the limit from either side of the limit.

Interpretation: This statement is false. Student A seems to be on the right track – that this statement depends on the function and x-value we are looking at. Interestingly, he/she has drawn what looks like a horizontal asymptote at y = 5. Is the student trying to make a connection based on the wording of the question, or because the limit is being viewed as related to an asymptote? Perhaps they are showing an example of a function that is getting arbitrarily close to y = 5, but does not pass this number? For Student B, it is hard to tell what he/she is thinking, especially without a drawing. They may be thinking about how the limit is not concerned about what is happening at x = a (based on a previous response not shown). Student C has given a statement about one-sided limits, perhaps thinking that if the limit approaches from both sides that the function must necessarily go there. Further questioning would be needed for  all students to really understand their thinking.

Question: How do you feel about the following statement “A limit is a number that the function value gets closer to but never reaches.”

Student A: A limit is a number that the function value gets closer to, but may or may not reach, depending on the function and x value chosen.
stuplot2.pngStudent B: True if a cannot equal x such as with an asymptote.
Student C: True because the function is undefined at the limit.

Interpretation: This statement is true, as the limit is unconcerned about what is happening exactly at x = a. Student A is perhaps equating f(a) with the limit of f(x) as x approaches a, which is a common misconception. Student B thinks the statement is true, provided that f(a) does not exist. However, the statement is true in general, even if f(a) is defined and is equal to the limit of f(x) as x approaches a. Making this disconnect is perhaps the most challenging piece conceptually with regards to limits. Student C perhaps is thinking of a removable discontinuity (hole); however, is falling into a similar thinking compared to Student B.

Question: How do you feel about the following statement A limit is an approximation that we can make as accurate as we wish.”

Student A: A limit may either be an approximation or an exact value, depending on the function and x value chosen.
Student B: False.
Student C: True.

Interpretation: The statement is false. Without any more information, we cannot make any judgement on the answers of Students B or C – they may only be best guesses. As for Student A, there is no separation of the limit from the approximation procedure used to illustrate the limit (often we try to illustrate the limit as the upper/lower bound of a sequence of values). One common misconception is to view the limit as a process of approximation when the limit is actually the upper/lower bound of this sequence of approximation values.

Question: How do you feel about the following statement “A limit can fail to exist at a certain point.”
Student A: True
Student B: True
Student C: True, when there are two points for the same x value

Interpretation: The statement is true. Student A and Student C have insightful responses. I believe Student C is thinking about a piecewise function where the two one-sided limits are not equal (even if the wording is a bit off).

Analysis and Conclusion

Overall, there does not appear to be much correlation with students success algebraically with limits and students conceptual understanding of limits. However, when we parse down to a more individual level, selecting a few students who show great success algebraically, we note that a couple misconceptions began to arise: (1) viewing the limit as an approximation, and (2) believing that the limit is somehow connected to f(a). There may be some confounding factors at play here. I am teaching calculus for the first time and my own explanations may not be as clear as a seasoned teacher. Also, the wording of the lab activity questions were taken directly from the paper above, and could have been difficult for first-year students to comprehend – more translation into easier-to-work-with statements could have bee done.

In conclusion, we have to ask ourselves what is it that we want of our students? Do we want students who are only able to work algebraically through the problems without any sense of the conceptual underpinnings? Do we want students who are conceptually strong, but lack algebraic skills? It is obvious that we want neither of these. So how do we attain students who show both algebraic finesse and conceptual understanding? We have to be respectful that algebraic proficiency and conceptual understanding pave a bi-directional road. One helps build the other, and we should spend ample time in each neighborhood (that’s a calculus joke). In fact, our students may learn better if we are able to combine the best of both approaches, without falling into the trap of promoting one type of knowledge over the other.

Social Identity within the Math Wars

When looking at the members of an ethnic group other than your own have you ever found yourself thinking “they all look the same to me.” This example illustrates the cognitive bias of out-group homogeneity, or the perception of individuals who do not fit into our identified in-group as less variable than they are. This way of thinking is particularly damaging in the context of the math wars for a couple reasons:

(1) Favorability within the in-group comes at the expense of individuals in the out-group.

One’s in-group may defined around a particular trait or dimension. In order to develop positive ties to the other group members, one must show that they too share the same trait or dimension. One way to accomplish this is to criticize an individual of comparable status within the out-group. The act of criticizing a member of the out-group reinforces one’s connection to the in-group, boosts a positive sense of self-esteem, and displays to other members that ideals are shared.

In the context of the math wars, let’s say we have two groups: a ‘traditional’ group and a ‘constructivist’ group. Whether or not these groups exist is the context of another discussion. For now, I will assume these rough boundaries, as I have certainly felt a divide in individuals over the past few years. As I entered my career in the math-education sphere, I recall (sadly) picking on individuals I deemed to be in the ‘constructivist’ group. At the time, I have to say it felt great. I was defining myself within my in-group at their expense. I remember I even tried calling out Boaler on one occasion (I was just as ambitious then apparently!). These call-outs did boost my favorability within my group. However, I failed to understand that those on the receiving end likely felt stupid, hurt and angry. And the main problem about arguing this way, is that both sides psychologically end up feeling stronger about the original differences, strengthening the divide between the two groups. I would like to take the time to own my previous interactions, and apologize for creating a space that was uncomfortable, unwarranted and unfair.

(2) Group-think. 

It becomes very easy to call-out an individual from the out-group when we perceive him or her to believe an ideal that our in-group disagrees with. As the divide between the groups grows stronger, we begin to define ourselves by believing “I am not like them.” For example, when I discuss students being fluent with math facts, I am often misunderstood to mean rote memorization. Note that ‘constructivists’ labeling me this way promote out-group homogeneity, and define themselves by not being one who believes in rote memorization. Continuous reinforcement within one’s in-group may lead to that moment when you are looking around the room and everyone is blindly nodding while the presenter admonishes the ‘traditional’ group for believing in rote memorization.

The honest truth is that I certainly do not believe that math facts require rote memorization (perhaps early coding of numbers, but this is a topic for another time). By definition, rote memorization means any comprehension of the meaning is removed. When I say fluency with math facts, I mean that comprehension and recall must work hand-in-hand. I do, however, believe that more recall of facts needs to be implemented in early grades – I want high storage and retrieval strength of these facts for later use! The act of labeling me as one who believes in rote memorization removes the possibility of bringing forth a more diverse conversation from a more diverse individual belonging to a more diverse group.

(3) Intergroup contact is prevented.

One possible way to dispel our own bias of out-group homogeneity is to spend time within the out-group (called intergroup contact). By speaking with individuals in the other group, we begin to understand their points of view, and they ours. That is, we begin to see them as a more diverse group than we had originally gave them credit for. Healthy conversation and movement forward is then possible. However, recall that individuals in a group tend to want to boost their self-esteem by looking more favorable within the group. Since these actions come at the expense of the out-group, the divide deepens and a cycle forms which may prevent intergroup contact.

I remember “scheduling issues” at my last place of employment. During my first year, my colleagues were very happy to allow me to volunteer my time in a classroom that we all would have labeled as more ‘traditional’. However, during the next school year when I was spending more time with an individual they deemed as ‘constructivist’ my timetable for the winter semester changed so that I was unable to volunteer any longer. The worst part was that I had developed a rapport with the local teachers, and we were all genuinely excited to be able to work together during the winter semester. I felt as though I had let them down – despite all of this being out of my control. In a similar vein, there were a few times within the past year in which members of the ‘traditional’ group nagged to determine my status within the group:

“I cannot tell which side you are on.”

“If you pursue a PhD, ensure you work on a question that will help our cause.”

These were probably some of the lowest lows I encountered. There is nothing quite like the feeling of your in-group consuming you whole, stripping you of your self-esteem, and leaving you to rot (yes, this is a fairly accurate description of how I felt at the time and sometimes still do). Psychologically I think this created a divide between me and those who identify as ‘traditional’, which caused me to look for social acceptance elsewhere (and also prompted a change in locale).

It has been challenging to build some of the rapport back with individuals whom I deem to be ‘traditional’ since psychologically I worry that all of this will happen again. However, it has been a worthwhile trip since I now feel the intergroup connectivity I have created. It turns out that the more time I spend with those who I had originally labeled ‘constructivist’ the more I see that this group is a largely diverse group. Sure, there are individuals that carry ideals that I disagree with – I don’t think this will change (and I don’t necessarily want it to). But the important lesson that I have learned is not to walk in with labels. Those individuals who would call themselves ‘constructivist’ are as diverse as those who would label themselves ‘traditional’. In fact, if we are able to look past our own out-group homogeneity biases, we may come to realize that our out-groups are more diverse than we think.

Baggage, Boxes and Beginnings

As many of you know a lot has happened in my life over the past few months. Really though, the story starts back two years ago when I accepted a position at a local university teaching jointly with the math & education faculties. Looking back, it certainly was one of the best and worst things to ever happen to me. It was great in the sense that it was a term position that acted as a stepping stone – working jointly for both math and education opened a lot of doors for me. And my ability to connect with many educators throughout the continent allowed my network to grow fairly quickly. For being such a political position, I did find it surprising that the university never used my networking skill set. I can’t say I am terribly surprised though since the university never did publish my name with the position title anywhere. Not even the minister of education at the time knew who I was, and the ministry was supplying some of the funding for the position (and this was in my second year of the term)! Very peculiar indeed. Anyway, back to the story at hand…

I remember fondly the week that I went out for lunch with two colleagues to try to gain a better picture of how they worked together. You see, one had a background in chemistry and the other had a background in science education. Their collaboration gave me hope that somehow there was a way for math folk and math education folk to get along, and I was sincerely interested in hearing their point of view. When the math department caught wind of this meeting, I was swiftly reminded that “my views do not represent the mathematics department.” I have carried these words as baggage with me for the past year and I am ready to let go. I never really talked about how these interactions made me feel since I was still employed by the university and never felt it was appropriate to voice anything publicly. Also, with two years left in the term position at the time, it didn’t seem right to get the union involved (you see the department put me in a very awkward conflict of interest, my direct supervisor being spouse to the chair of the department) so I didn’t really have anyone to talk to. There was even a point where I hit such a low that I called a counselling service provided by the university because I just didn’t have anyone to talk to about how I felt.

Piled on top of this baggage I was carrying, were my usual teaching duties, as well as a few other items that turned  this into a hectic time for me personally. Two of my grandparents died, and the grief of losing valued members of a close-knit family hovers around you for some time. David proposed, which meant a social (a large party-like event held before the wedding) and a wedding to plan. My house got broken into twice, with several thousands of dollars of items stolen, which added to already high stress levels. One of my blogs was deemed a personal attack when I was trying to reflect on and question my own teaching biases. My health degraded, and there were several nights per week in which I was unable to sleep, resulting in me having to string together long days of teaching. Of course, I became even more stressed out and irritable during these times due to my digestive issues. Three doctors later and all we could come up with was IBS and a diet of no milk, eggs, gluten, nuts, mushrooms, alcohol, or soy. I was pretty much the worst house-guest to have over for dinner. #LentilsForMonths

To me, the worst part was feeling a lack of a sense of belonging. Every day I felt physically and emotionally hurt. I knew my students appreciated me, and that the education faculty was fine with how I was treating their students. But the courses I taught were specifically for the math department, and I just felt disconnected from them all the time. The closest I felt to them over the past year was when I told them that I had found a new job! Seriously, what the heck?! Reflecting a bit over the past year, I can see a subtle swing in my perceptions. As I became somewhat black-listed by the traditional group, I became more close with individuals the traditional camp might label constructivist. “What’s in a name?” I thought. Despite previous transgressions, they still welcomed me with open arms, saying “Hey, let’s think about how to best teach math to our students together.”

Together. Now that was a word I had desperately been longing for. I was able to reflect and ask questions without fear of getting the stink eye if I didn’t phrase my answer using worked examples or testing (that was a joke…  but not not really a joke). It was refreshing. And most importantly, it felt like I had a sense of belonging. In some sense, it felt like a new beginning was on the horizon. And then June happened.

With a wedding on June 11th, I remember getting that phone call to fly in for an interview on  June 8th. I packed my bag, flew out, attended the interview and flew back home all in the span of 24 hours. David and I continued preparing for our wedding, and had a beautiful weekend with close friends and family. He swept me off my feet… literally.View More:

Then Monday rolled around and I received word that I, being the underdog of the short-list, had somehow managed to land the position. A special shout-out to this Desmos activity which I think really allowed me to shine over the other candidates. It has been a summer from some kind of fairy-tale: new friendships and collaborations with amazing people in the MTBoS community, an opportunity to shine in B.C., and a new adventure with the man of my dreams. Oh, and boxes. Lots of boxes. If I never have to see another box for the rest of my life, I will be relieved.

I am happy to report that, while my life is currently fragmented and packed away in the back of a loading truck, I have never felt more whole. Over the past couple months, my health has returned, much to the delight of my taste buds. I was asked to present at researchED Washington in the fall. Heck, David and I even hung wallpaper in our back room and didn’t file for a divorce. I think things are finally starting to look up! Thank you to all of you whom I have confided in, and those of you who helped me see the light even in the darkest times – your presence has truly been appreciated. I have one more blog post on the way that is more research-oriented and reflective, but then I am going to take a bit of a break in posting over the fall, as I have a house to unpack, courses to prep, two conferences to speak at and potentially a few online collaborations to think about.

Survivorship Bias in Education

An Interesting Example

During World War II, Abraham Wald worked for the New York company Statistical Research Group. The American military approached him with a problem. War planes were coming back from air battles covered in bullet holes, and the military was interested in protecting future planes by using minimal armour placement. The question was where to best place this armour so as to protect the planes and pilots. The bullet holes were roughly distributed as such:UntitledWhere would you choose to place the minimal armour?

If you said any section other than the engine section, then you would be absolutely… wrong. Wald’s explanation is as follows: the statistics that were presented were gathered from planes that had survived battle – the planes that didn’t survive battle were not included in the numbers. Thus, the planes that returned were not a random sample and conclusions could not be made by only reflecting on the surviving planes. Wald recommended adding armour to the sections of the plane that returned home relatively unscathed (here, the engine section) would be wise, as the surviving planes tell us the regions of the plane that can take damage and still return home safely.

Suggesting to give armour to regions of the plane that are damaged the most reflects a statistical bias known as survivorship bias. This is an error that arises when we only consider objects that “survive” a particular process, ignoring the objects that do not survive. For example, in In Search of Excellence, Peters & Waterman observe 43 “excellent” companies and determine eight traits that have propelled these companies to the top. However, as of 2016, 20 of the 35 public traded companies were under market average, and five of these companies have since gone bankrupt. So what’s going on here? Are these companies actually “not excellent”? Well, there are other factors at play here, such as luck, that may have helped these companies propel themselves to the top. And since Peters & Waterman only chose companies they deemed as “excellent”, their sample reflected survivorship bias: maybe some of the poorest-performing businesses also used these eight traits – we don’t know since only “excellent” companies were analyzed.

Survivorship Bias in Education

The above aforementioned is why I typically don’t care too much for the advice of celebrities. Why would I listen to their advice on how to be successful when they are a part of the small minority that have survived the process? How do we know there aren’t other factors such as luck or socioeconomic status that have helped advance their careers? What about all of the people who have tried to become a celebrity by following this advice, but haven’t survived the process?

So why is it that we listen to similar advice of celebrities in mathematics education?

Arguments such as “I never memorized my times tables and it’s never held me back.” reflect survivorship bias. Maybe the statement is true, and that’s fine, but what is missing are the countless number of individuals who did not memorize their times tables and have been held back in some form. We need to see and hear those stories to get a full picture on what to conclude. In addition to the previous argument, award-winning mathematicians who “think slowly” and “visually” are often given as key examples in pedagogical arguments. What about all the mathematicians who are not visual thinkers, or who don’t think slowly? Are there other factors, such as the field of study, that cause these traits? Again, more examples are needed to get a better sense of the typical traits of a mathematician.

Concluding Remarks

As educators, I think we get so excited when we hear about the stories of progressive-style pedagogy that works, that we often forget that there may be just as many stories in which progressive-style pedagogy doesn’t work. Traditional-style pedagogy isn’t safe from survivorship bias either – how many times have you, within a traditional-style lesson, received a correct answer from one student and proceeded through the lesson, oblivious to the fact that there may be several students confused? You are making an assumption based on the “survivors” and your “non-survivors” are potentially being left behind. We need to be mindful of this, and not allow survivorship bias to elude us into thinking something is there when it is not.

I want to ask a favour of all the teachers out there: I want to know of a time when you utilized progressive-style pedagogy and it failed. Personally, I had one class in which I flipped my lessons and it was a disaster – with pass rates being abysmally lower than normal. This came after I was on a high from the last semester, in which I flipped my class and pass rates were much higher than normal! There were definitely differences between the two cohorts of students that I had failed to see until post-reflection.

Interestingly enough, Peters & Waterman, the authors of In Search of Excellence, later came out and said that they had made up some of their data. I firmly believe that both sides of the math debate have valuable advice to offer, which is why I am interested in hearing the different stories out there. I do get concerned, however, when teachers/researchers begin to fall into the Peters & Waterman trap: “… go find smart people who are doing cool stuff from which you can learn the most useful, cutting-edge principles … [and] worry about proving the facts later.” Rather, let’s observe all sides of our pedagogy (both the good and the bad) and use this information to perfect our craft.