A few weeks back, I attended ICER 2013 at UC San Diego. Afterwards, I went up to San Francisco and had some adventures there (and at Yosemite), and then spent time in Vancouver seeing friends before coming home.

ICER this year was a solid conference, as always. I liked that this year things reverted to having two minute roundtable discussions after every talk, before the Q&A. It makes for a much more engaging conference.

All hail the UCSD Sun God, who benevolently oversaw the conference.

My favourite talk this year was definitely Tom Park et al’s “Towards a taxonomy of errors in HTML/CSS“. HTML/CSS tends not to get studied in the CS ed crowd, and as Tom’s talk illustrated, that’s a big shame. HTML is for many people the first (and sometimes only) formal language that they encounter.

It turns out that many of the same stumbling blocks that people have when learning programming languages are the same as when they learn HTML. Syntax errors, figuring inconsistent syntax, learning that things need to be consistent – even just learning that what you see is not what you get.

In compsci we tend to overlook teaching HTML since it’s a markup language, not a programming language. But what we deal with in compsci is formal languages, and the simplest ones are the markup languages. Playing with a markup language is actually a much simpler place to start than giving novices a fully-fledged, Turing complete language.

Other research talks of note:

• Peter Hubwieser et al presented a paper on categorizing PCK in computer science that I liked; I’d love to see more work on PCK in CS, and look forward to seeing subsequent work using their framework.
• Colleen Lewis et al performed a replication study looking at AP CS exams. I love replication studies, so I may be a bit biased towards it :) In the original paper, they found that the first AP CS exam’s scores were strongly predicted by only a handful of questions – and those questions were ones like:
  int p = 3
  int q = 8
  p = q
  q = p_
  what are the values of p and q?_
  In Colleen’s paper, they found that the newer AP CS exams are much more balanced: things are not predicted by a small number of questions. Good to see!
• Robert McCartney et al revisited the famous McCraken study that found that students can’t program after CS1, and found that students can, but that educators have unrealistic expectations for what students would know by the end of CS1.
• Michelle Friend and Robert Cutler found that grade 8 students, when asked to write and evaluate search algorithms, favour a sentinel search (go every 100 spots, then every 10, then then every 1 spots, etc) over binary search.* Mike Hewner found that CS students pick their CS courses with really no knowledge of what they’ll be learning in the class. It’s one of those findings that’s kind of obvious in retrospect, but we educators really tend to mistake our students as thinking they know what they’re in for. Really, a student about to take a CS2 class doesn’t know what “hash tables” or “graphs” are coming in. Students pick classes more around who the prof is, the class’ reputation, and time of day.

• Finally, Michael Lee et al found that providing assessments improve how many levels people will play in an educational game that teaches programming. It’s a neat paper, but the finding is kind of predictable given the literature on feedback for students.

  I much more enjoyed Michael’s ICER talk from two years ago. He found in the same educational game that the more the compiler was personified, the more people played the game. Having a compiler that gives emoticon-style facial expressions, and uses first person pronouns (I think you missed a semi-colon vs. Missing semicolon) makes a dramatic difference in how much more people engage with learning computing. That’s a fairly ground-breaking discovery and I highly recommend the paper.
  The conference, was of course, not only limited to research talks:

• The doctoral consortium, as always, was a great experience. I got good feedback, about a dozen things to read, as well as awesome conference-buddies!

• The lightning talks were neat. My favourite was Ed Knorr’s talk on teaching fourth-year databases, since third-year and fourth-year CS courses are so often overlooked in the CS ed community. I also liked Kathi Fisler’s talk on using outcome-based grading in CS.
• The discussion talks were interesting! Elena Glassman talked about having students compare solution approaches in programming, which nicely complements the work that I presented this year.* The keynote talk also talked about the importance of comparing-and-contrasting. My takeaway from the keynote, however, was a teaching tip. Scott, the speaker, found that students learnt more from assignments if they were asked upon hand-in to grade themselves on the assignment. (Students would basically get a bonus if their self-assessment agreed with the TAs’ assessment.) It’s such a small thing to add onto the hand-in process, and adds so much to how much students get out of it. I’ll definitely have to try this next time I teach.
  Overall, a great time, and I’m looking forward to ICER 2014 in Glasgow!

At today’s CS Education Reading Group, one of our group members led us through an exercise about group work from “Students’ cooperation in teamwork: binding the individual and the team interests“ by Orit Hazzan and Yael Dubinsky.

It’s an in-class activity to get students thinking about how they work together in software projects. Students are given a scenario: you’ll be on a software team. If the project completes early, the team gets a bonus. How should the bonus be allocated?

1. 100% of the bonus should be shared equally
2. 80% should be for the team to share; 20% should go to the top contributor
3. 50% team, 50% individual
4. 80% team, 20% individual
5. 0% team, 100% to the individual who contributed the mostEverybody in the room got a minute to say which option we’d prefer and to write it down – and then we had a discussion about it. We then went through the rest of Orit’s paper and the variant scenarios.

I was the sole person in the room arguing for 100% team. My reasoning was because individual bonuses are not effective rewards – and often counterproductive.

Large Monetary Awards are Counterproductive

Ariely et al found that larger monetary rewards actually reduce performance on cognitive, intellectual tasks (link).  There’s almost half a century of psychology research arguing this.

And it doesn’t just hold in laboratory studies – though it’s not as simple as in the lab. Pay-for-performance approaches in the white-collar workplace have been repeatedly found to be at least suboptimal.

External motivators generally don’t help with cognitive tasks – internal motivation is really what drives us to do well on cognitive tasks.

Bonuses and Justice

Another problem with bonuses is fairness. Women and a number of other minorities are less likely to get them. They’re less likely to argue they deserve them. Their contributions are more likely to be viewed as less important. And they are perceived as less valuable.

(On that note, tipping in restaurants in the like is known to amplify racial inequalities. The race of the server is a stronger predictor of gratuity size than the quality of the service.)

Student Perceptions of Teamwork

In Orit’s small-sized classes, students opted for 80% team, 20% individual (see her 2003 ITiCSE paper). Why not 100%? One of the things that came up in our discussion is the question “but who is on my team?”

For a lot of our discussants, team composition was the driving factor. Do you have a team you trust? Then 100% for the team, for sure. But what if you don’t know them? Or you don’t trust them?

Katrina Falkner et al did a study on how CS students perceive collaborative activities, which they presented at last year’s SIGCSE. For a lot of students, collaboration stresses them out: they’re not used to it, they’re not experienced at it, and they’re not particularly good at it. But as educators, that’s what we’re here to work on, right?

The biggest source of anxiety for students in Katrina’s study was in who their partners were/would be. Would their partner(s) be smart, hardworking, and reliable?

Team Composition

It turns out randomized groups were the worst for students. Students felt powerless over their performance. We know from other literature that randomized groups is suboptimal for student performance. A much better way to form groups for performance is to group students by ability – strong students with fellow strong students.

On that note – it can be disastrous to pair strong students with weak students on the idea that poor students learn from the weak ones. It seeds/reinforces a lot of student fears about group work: strong students dislike it as they have to do a disproportionate share of the work; weak students learn less as their partner is doing the work for them.

Moving on: the best way to form groups in terms of reducing student anxiety is often to let students pick their groups. I say “often” because for a student who feels like an odd-one-out in the class, or doesn’t know anybody, this can be just as stressful.

Managing Short Term Stress

Stress is another thing worth talking about. Some people do great under pressure, and work better with the focus it gives them. And some people fall apart under stress, and work best without pressure. (And most of us are somewhere in between.)

The good news is that how we interpret anxiety is fairly malleable, and in a good way:

_The first experiment was at Harvard University with undergraduates who were studying for the Graduate Record Examination. Before taking a practice test, the students read a short note explaining that the study’s purpose was to examine the effects of stress on cognition. Half of the students, however, were also given a statement declaring that recent research suggests “people who feel anxious during a test might actually do better.” Therefore, if the students felt anxious during the practice test, it said, “you shouldn’t feel concerned. . . simply remind yourself that your arousal could be helping you do well.” _

Just reading this statement significantly improved students’ performance. They scored 50 points higher in the quantitative section (out of a possible 800) than the control group on the practice test. 

– Why Can Some Kids Handle Stress While Others Fall Apart?

Getting that message out to students is something we ought to be doing – test anxiety hurts a lot of student, as does anxiety about group work. It doesn’t have to be so bad.

In five days, I’ll be heading to ICER 2013 in San Diego, where I’ll be presenting a paper “Comparing and Contrasting Different Algorithms Leads to Increased Student Learning“ (pdf here).

The findings in a nutshell: if you present two different approaches to solving a compsci problem side-by-side and have students compare them, the students will understand the problem better than if you present those approaches sequentially. And importantly, the students will be better transferring their understanding of the problem to similar problems.

Why is this notable? Because the sequential approach is pretty much ~95% of how we teach algorithms and data structures! Just this past term when teaching CS2 I did things this way: a unit on binary trees, then a unit on BSTs, then a unit on heaps. Yet the evidence we have here is that it’s better to show algorithms/data structure variation in parallel.

Background Knowledge

This study is a replication of two studies in math education –  a study on algebra problems, and a fol]low-up study on estimation problems, both by Bethany Rittle-Johnson and Jon R. Star.

In the original algebra study, students were randomly assigned to one of two groups:

• a control group where students were given workbooks that had students saw a worked example solving a problem using one approach; they answered questions about it; then on the next page they saw a second worked example solving the problem using a different approach, and answered questions about it

• an experimental group where the workbooks presented the two worked examples side-by-side and students worked on all those problems on the same page
  The study has a pretest-intervention-posttest model, where the pretest and posttest are the same test. This allows the researchers to see how much students learnt from the intervention. The tests probed three things:

• procedural knowledge – how to solve a problem* procedural flexibility – being able to solve a problem with multiple approaches; being able to generate, recognize and evaluate different approaches

• conceptual knowledge
  And what did they find?

• the compare & contrast group did better than the sequential group with respect to procedural knowledge and procedural flexibility

• the two groups did the same on conceptual knowledge
  The conceptual knowledge thing was a bit of a surprise to them. So, they did another study! This time, they did a prestest-intervention-posttest-followup study. That’s the same as before, but with a second posttest given some time after the study, to see how much students retain. In this study, the math problems were about estimation.

What did they find?

• Again, the compare and contrast group did better on the posttest and the follow-up with regard to procedural knowledge and procedural flexibility

• But again, the two groups are the same on conceptual knowledge.
  It bothered them some more that the conceptual knowledge was different. Some similar studies in other fields would lead one to predict conceptual knowledge would be the same. So, they looked closer at their data:

• For the students who started off with some conceptual knowledge, the compare & contrast condition lead to more learning.

• For the students who had no conceptual knowledge to begin with, it didn’t matter which condition they were given.They speculated that you need to have some existing knowledge of a problem for comparing and contrasting to really shine.

Our study

We ran our study as a pretest-intervention-posttest-followup study, following the estimation study. In our study, CS2 students compared different techniques for hashing. We ran the study in three different sections of CSC 148 at UToronto, with 241 students participating.

Not that surprisingly, students in the compare-and-contrast group performed better at procedural knowledge and procedural flexibility – and the two groups performed the same on conceptual knowledge.

But we found the opposite of Rittle-Johnson and Star when we looked closer at conceptual knowledge:

• students who started with _no_ conceptual knowledge gained more from the compare-and-contrast condition than from the sequential condition
• students who started with some conceptual knowledge performed the same in both conditions
  What we think is going on here is that the compare-and-contrast approach lets students build a schema. By looking at what is different and similar, they can decipher what the important aspects of a problem are.

For the students who already have such a schema in place, when they see things sequentially, they can relate things back to their schema. They already know what the important aspects of a problem are, and so can compare on the fly.

For an expert like me, or any other instructor, this is the same for us. When I look at a hash function, I already know the aspects of hashing that can be varied to solve problems differently. When I see something new presented, I can relate it back to what I already know.

Another difference comes with yardsticks. Experts work with yardsticks like big O notation and complexity classes – that allow us to scale up our knowledge very easily. For novices, it’s a lot easier to handle “mergesort is faster than selection sort” than “mergesort is O(nlgn)”.

For us experts, it makes sense to present information sequentially – because we can easily process things that way. For our students – the ones learning things completely afresh – that’s a lot harder.

This came up at /r/CSEducation today, and I thought I’d summarize the literature I’ve seen regarding Black/Hispanic enrolments in computer science in North America. What factors do we know to be behind the lower numbers of Black/Hispanic students in North American CS classrooms?

It’s a multi-part problem: fewer Black/Hispanic students show up to begin with – and then they’re less likely to graduate with a CS major at the end of their university career. I’ve broken up the factors I’ve seen in the literature based on_ _when in the “leaky pipeline” they most apply.

I’m aiming here to give a quick-and-dirty overview of the issues – there’s a fair bit of literature on this and the references below provide an excellent place to start on the literature.

(Sidenote: The Varma paper ([2]) also looks at Aboriginal students; my impression from the few Aboriginal CSers I know is that they parallel many of the same issues. There is unfortunately very little research on First Nations, Metis and Inuit under-representation in computer science.)

The Leaky Pipeline: Middle School

1. In middle school, Black and Hispanic youth are just as interested in computer science as their White and Asian peers. [1, 2]
2. Black and Hispanic youth are less likely to have a computer at home [1, 3].
3. For White boys, video games are where many of them first “pull back the curtain” on how computers work. But while Black boys play just as much video games as White boys, modding and cheat codes aren’t part of their gaming cultures – and don’t hence “pull back” the curtain [3]. They don’t have the “privilege to break things“.
4. Characters in video games have a lack of racial diversity [3] – from a young age Black and Hispanic students imagine computer scientists as “White or Asian men”; computer science does not seem relevant to them.

High School

1. Black and Hispanic students are more likely to go to disadvantaged k-12 schools [4, 5]. 
2. They’re less likely to graduate from high school than their white peers, and lower expectations are placed on them [1, 4]. 
3. And for those that do succeed, they’re less likely to have a high school CS class available to them. The situation has actually been getting worse with the testing movement – disadvantaged schools are removing CS since it’s an “extra”, and they have a hard time recruiting/retaining qualified teachers [4].

Choosing to Study University CS

1. Encouragement is really, really important. And Black/Hispanic students are less likely to be encouraged by parents, guardians, teachers, or peers to study computer science [2, 6, 7]. Encouragement has a stronger effect on students than their ability at computer science [6] – and has the potential to overcome differences in preparation for university CS.
2. Black and Hispanic girls are less likely than their White peers to know somebody who works in STEM, and are less likely to have parents in STEM. [2]
3. Black and Hispanic youth are more aware/worried about gender/racial discrimination in STEM than their White peers [2, 7].
4. Black and Hispanic students are motivated to study computer science because it is a prestigious, secure career, and provides social status [2, 5, 6, 8]. While they are turned on by the creative, pro-social, problem solving part of computer science – and are more engaged when CS is taught that way – they feel like “do what you love” is a luxury for rich White people [5].5. Black, Hispanic and low-class White women choose universities differently than middle/upper-class White women. The latter care about things like reputation and programme detalis. The former care about tuition, scholarships, and closeness to family [5]. At my university, tuition is higher for computer science than it is for other Arts & Science majors. We’re likely not doing any favours to diversity here. 

Staying in CS Majors

1. When Black and Hispanic students do show up to university CS, they are more likely than their White and Asian peers to feel underprepared. Indeed, 48% of Black, Hispanic and Aboriginal students feel not prepared “at all” [5].
2. I’m gonna repeat it since it bears repeating: _Encouragement has a stronger effect on students than their ability at computer science [6] – and has the potential to overcome differences in preparation for university CS._3. The heavy workload in CS courses is a problem for many of these students. You need to be “unmarried, single, no kids, no job, no hobbies, no dependents” [5]. Black and Hispanic students are disproportionately likely to be “non-traditional” students (have families, mature students, etc). Many Black/Hispanic students will leave CS because of the workload [5]. One contributing factor is social habits: whereas Asian students are likely to study together as part of their social life, Black students are more likely to study in isolation and not as part of their social life [8].
3. Another major reason they leave is hostility. They find they can’t be taken seriously due to their race (and gender, if a woman on top of it) [2]. And they’re more likely to feel like “outsiders” in CS [1]. Though they feel like outsiders, it’s worth noting that lack of identification as a geek/nerd appears not to be an issue [5].
   
   Some things that can be done:

• Improve CS outreach to disadvantaged schools. Early encouragement and exposure to CS is important [7].
• Lobby to make CS mandatory in high school for everybody.* Promote co-op in CS programmes, and appeal to the fact that CS offers a solid career path. Co-op has the advantage of helping with tuition – another concern for non-Asian racial minorities [5].
• Improving scholarship opportunities for underrepresented minorities and low-income students to study computer scinece.
• Provide mentoring programmes for students, like the Tri-Mentoring Programme at UBC – mentorship is a valuable source of encouragement and advice for students of underrepresented groups.
• Provide social learning communities, like the First-Year Learning Communities we have at U of Toronto – make it part of CS students’ social lives to study together.

References

1. Zarrett, Nicole, et al. “Examining the gender gap in IT by race: Young adults’ decisions to pursue an IT career.” Women and information technology: Research on underrepresentation (2006): 55-88.
2. Girl Scout Research Institute. “Generation STEM: What Girls Say about Science, Technology, Engineering and Math“. (2012)
3. DiSalvo, Betsy James, Kevin Crowley, and Roy Norwood. “Learning in Context Digital Games and Young Black Men.” Games and Culture 3.2 (2008): 131-141.
4. Goode, Joanna, Rachel Estrella, and Jane Margolis. “Lost in translation: Gender and high school computer science.” Women and information technology: Research on underrepresentation (2006): 89-114.
5. Varma, Roli. “Women in computing: The role of geek culture.” Science as culture 16.4 (2007): 359-376.
6. Guzdial, Mark, et al. “A statewide survey on computing education pathways and influences: factors in broadening participation in computing.“ Proceedings of the ninth annual international conference on International computing education research. ACM, 2012.
7. Zarrett, Nicole R., and Oksana Malanchuk. “Who’s computing? Gender and race differences in young adults’ decisions to pursue an information technology career.“ New directions for child and adolescent development 2005.110 (2005): 65-84.
8. Margolis, Jane, and Allan Fisher. Unlocking the clubhouse: Women in computing. The MIT Press, 2003.

As somebody interested in gender equality in CS, one thing that’s proved quite illuminating for me is to read up on the psychology of sexism. Why does sexism persist in society? What social and psychological structures keep it in place?

Sexism in some ways is unlike other forms of discrimination. When it comes to race, or class, or disability, the social psychology literature will frequently talk about social distance. But when it comes to women, men “can’t live without ‘em” [1] – and so things tend to be a bit different.

Ambivalent Sexism

It turns out that sexism has two faces: good old hostile sexism – and the more palatable benevolent sexism. Benevolent sexism is the notion that women are wonderful, caring, nurturing and beautiful creatures – and so must be protected and provided for. (Note the “creatures” – not “people”.)

The evidence on the psychology of sexism is that the people who espouse hostile sexism are also benevolently sexist. They think women shouldn’t work – women should stay home and care for the children because women are so good at mothering. And they get hostile when their regressive worldview clashes with attempts of women and men to change the status quo.

Furthermore, the more you’re exposed to benevolent sexism, the more likely you are to later take on hostile sexist views. The more you’re primed to believe that women should fill the magical-traditional role, the more likely you’ll try to thwart any attempt to move away from this view.

A vital thing to note about benevolent sexism is how frequently it is embraced by women. Many women will argue that women are better at being nurturing, or communicating, etc. After all, why would you turn down a worldview that argues you’re wonderful and worthy of protection?

Hammond et al’s 2013 “The Allure of Sexism: Psychological Entitlement Fosters Women’s Endorsement of Benevolent Sexism Over Time“ gives the most up-to-date review I’ve seen on ambivalent sexism in its introduction – the rest of this paper is largely a summary of their literature review.

Benevolent Sexism

Benevolent sexism tends to be a fairly palatable type of sexism since it doesn’t seem sexist [2] – indeed women often see it as chivalry or intimacy rather than sexism [3]. After all, women “complete” men and are their “better halves”.

While hostile sexism “works to suppress direct challenges to men’s power by threatening women who taken on career roles or seek political reform” [2], benevolent sexism works to incentivise women’s adoption of traditional, patriarchal gender roles. Women are revered as special and caring – and deserving of protection.

And indeed, the main way in which benevolent sexism stops gender inequality is through women’s adoption of these views. Benevolent sexism incentizises women to stay in those special, caring, protection-worthy gender roles – and at the same time makes men’s social advantages seem more fair [4].

Effects on Women

A longitudinal study of New Zealanders found that psychological entitlement in women leads to greater endorsement of benevolent sexism. (Hammond et al, 2013). The more a woman believes she is deserving of good things, praise, and material wealth – the more benevolent sexism will appeal to her. After all, benevolent sexism both reveres women as being men’s “better halves” – and at the same time promises women that they will be protected and financially provided for by their male partners.

Women who espouse benevolent sexism have greater life satisfaction [5]. They feel good about their focus on care-giving and appreciate being provided for by their male partners. They have more power to influence their male partners [6], and hence have indirect access to resources and power. 

At the same time, however, benevolent sexism holds women back when it comes to gaining social power. Women who hold benevolently sexist beliefs have less ambition for education and their careers [7]. They are more likely to defer to their partners on career decisions [8]. They believe their careers should take the back seat to their male partner’s careers [9].

And furthermore, exposure to benevolently sexist statements leads women to perform poorly at tasks and feel lower competence [10]. They’re more likely to believe that men and women have an equal chance of success in society [4]. And they’re less likely to support gender-based collective action [11].

Effects on Men

For men, espousing benevolent sexism requires sacrifice in relationships [12]. They have to provide for their female partners, and live up to the romantic notion of the ‘’white knight’’.

But this sacrifice does come with benefits. Men who are portrayed as benevolently sexist are viewed as more attractive [13]. And men who actually agree with benevolent sexism are more caring and satified relationship partners [6].

Furthermore, women who espouse benevolent sexism are likely to be hostile/resistant to men who aren’t on the same page [6]. And women who espouse benevolent sexism are more let down when their male partners fail to live up to the fantasy of the white knight [14].

A Tragedy of the Commons

For women, benevolent sexism gives them dyadic power [1]. They have more power in relationships, and more satisfaction in them. As a result, women have something to lose if the status quo is disrupted [12].

But getting the individual benefits of benevolent sexism means agreement with the attitudes that perpetuate gender inequality. The women who agree with benevolent sexism are more likely to hold themselves back in their careers and less likely to support feminist causes.

If we want to get more women into non-traditional careers like computer science, sexism is something we need to tackle. We spend a lot of time trying to break down hostile sexism in STEM – but what about benevolent sexism? 

References

1. Glick, Peter, and Susan T. Fiske. “Hostile and benevolent sexism measuring ambivalent sexist attitudes toward women.” Psychology of Women Quarterly 21.1 (1997): 119-135.
2. Hammond, Matthew D., Chris G. Sibley, and Nickola C. Overall. “The Allure of Sexism Psychological Entitlement Fosters Women’s Endorsement of Benevolent Sexism Over Time.” Social Psychological and Personality Science (2013): 1948550613506124.
3. Barreto, M., & Ellemers, N. (2005). The burden of benevolent sexism: How it contributes to the maintenance of gender inequalities. European Journal of Social Psychology, 35, 633-642. doi:10.1002/ejsp.270
4. Jost, J. T., & Kay, A. C. (2005). Exposure to benevolent sexism and complementary gender stereotypes: Consequences for specific and diffuse forms of system justification. Journal of Personality and Social Psychology, 88, 498–509. doi:10.1037/0022-3514.88.3.498
5. Hammond, M. D., & Sibley, C. G. (2011). Why are benevolent sexists happier? Sex Roles, 65, 332–343. doi:10.1007/s11199-011-0017-2
6. Overall, N. C., Sibley, C. G., & Tan, R. (2011). The costs and benefits of sexism: Resistance to influence during relationship conflict. Journal of Personality and Social Psychology, 101, 271–290. doi:10.1037/a0022727
7. Fernandez, M., Castro, Y., Otero, M., Foltz, M., & Lorenzo, M. (2006). Sexism, vocational goals, and motivation as predictors of men’s and women’s career choice. Sex Roles, 55, 267–272. doi:10.1007/s11199-006-9079-y
8. Moya, M., Glick, P., Exposito, F., de Lemus, S., & Hart, J. (2007). It’s for your own good: Benevolent sexism and women’s reactions to protectively justified restrictions. Personality and Social Psychology Bulletin, 33, 1421–1434. doi:10.1177/0146167207304790
9. Chen, Z., Fiske, S. T., & Lee, T. L. (2009). Ambivalent Sexism and power-related gender-role ideology in marriage. Sex Roles, 60, 765–778. doi:10.1007/s11199-009-9585-9
10. Dardenne, B., Dumont, M., & Bollier, T. (2007). Insidious dangers of benevolent sexism: consequences for women’s performance. Journal of Personality and Social Psychology, 93, 764–779. doi:10.1037/0022-3514.93.5.764
11. Becker, J. C., & Wright, S. C. (2011). Yet another dark side of chivalry: Benevolent sexism undermines and hostile sexism motivates collective action for social change. Journal of Personality and Social Psychology, 101, 62–77. doi:10.1037/a0022615
12. Glick, P., & Fiske, S. T. (1996). The ambivalent sexism inventory: Differentiating hostile and benevolent sexism. Journal of Personality and Social Psychology, 70, 491–512. doi:10.1037//0022-3514.70.3.491
13. Kilianski, S. E., & Rudman, L. A. (1998). Wanting it both ways: Do women approve of benevolent sexism? Sex Roles, 39, 333–352. doi:10.1023/A:1018814924402
14. Hammond, M. D., & Overall, N. C. (2013). When relationships do not live up to benevolent ideals: Women’s benevolent sexism and sensitivity to relationship problems. European Journal of Social Psychology, 43, 212–223. doi:10.1002/ejsp.1939

This term, I’m taking my supervisor’s grad course on “Systems Thinking for Global Problems“. It’s been quite interesting so far. In our last couple of lectures, we have been talking about feedback loops.

And with that on my mind, I was particularly struck by a recent post on Mark Guzdial’s blog reposting a keynote by Eric Roberts:

[in response to increasing CS enrolments], 80% of the universities are responding by increasing teaching loads, 50% by decreasing course offerings and concentrating their available faculty on larger but fewer courses, and 66% are using more graduate-student teaching assistants or part-time faculty. […] However, these measures make the universities’ environments less attractive for employment and are exactly counterproductive to their need to maintain and expand their labor supply. They are also counterproductive to producing more new faculty since the image graduate students get of academic careers is one of harassment, frustration, and too few rewards. 
Computer science departments have, for decades, had cyclical enrolment. The sort of oscillation in enrolments is exactly the sort of thing you see in systems analysis when you have a balancing feedback loop with a delay in it.

Balancing Feedback Loops

Causal loops are used in systems analysis to show the relationship between variables in a system. If the variable _x_ increases when _y_ increases, and _x_ decreases when _y _decreases, they have a positive link. If, however, _x_ increases when _y _decreases, and _x_ decreases when _y_ increases, they have a negative link.

Let’s say we put a bunch of variables in a loop. If there’s an even number of negative links, then we have a reinforcing feedback loop: the system will increase (or decrease) exponentially until it hits some limit to growth. The negative links cancel each other out – so everything just reinforces everything.

But what if we have an odd number of negative links? The system tends towards an equilibrium – either it will asymptote to some value, or, more often, it will oscillate. Something will increase, another thing will decrease it, another will increase, and so on.

Consider:

As the number of students enroling in CS1 increases, the quality of student experience in a CS programme goes down for the reasons Eric Roberts covered above. And as the quality of the student experience goes down, the CS enrolments go down.

Eventually, enrolments and my abstract “student experience” will reach equilibrium – but it won’t be a static one. The enrolments will oscillate due to delay in the system: when CS enrolments increase, the quality of “student experience” won’t go down for a while yet, and “student experience” won’t immediately affect CS enrolments.

### Playing with the Model

Where do CS1 enrolments come from? Eric Roberts has elsewhere observed that CS1 enrolments at Stanford (and elsewhere) are positively correlated with the NASDAQ average – with some delay.

And with some delay, one would think (hope?) that the number of computer scientists in the economy would increase the NASDAQ average. Again, a balancing loop emerges with some delays in it:

Let’s think some more about the relationship between CS1 enrolments and the number of CS graduates.

The more CS1 enrolments there are, the more students in a CS department relative to the number of CS professors. Let’s call that “students / profs” for short.

As students / profs increases, the use of sessional lecturers increases. The number of courses a department offers decreases – which in turn decreases the amount of streaming in CS programmes. The teaching load for faculty increases, in turn hurting faculty satisfaction.

Even with the abstract “student experience” unpacked somewhat, we still see a balancing feedback loop. It doesn’t matter what path you take from “students/profs” to student retention – there’s an odd number of negative links.

Now, everything so far has assumed the number of faculty is fixed. But it’s not quite – with a (often substatial) delay, increased enrolments will lead to more faculty hirings. Let’s look a bit more at that:

Here we have our first reinforcing loop. The more students/profs, the more the teaching load – and down goes faculty satisfaction, more profs quit and go into industry, and then the ratio of students/profs gets even worse.

This feedback loop would continue on until it hits a limit to growth (no faculty to teach classes?) if it weren’t for the interacting effect of faculty hirings. The more faculty leave, the more faculty need to be hired. If we ignore our link between faculty turnover and the number of faculty, what we have here is a balancing feedback loop: profs who leave are replaced, and all is steady.

What could change that is university funding for hiring more faculty above the replacement rate. This is going to be institution-specific, so it’s hard for me to come up with a model here. (Even for any given institution, funding structures tend to be incredibly complicated.)

As I’ve been playing with these models, it’s striking me that it’s unlikely the cyclical enrolments in CS will stop. For them to stop, we’d have have either a nice steady tech economy – meaning interest in CS was steady – or we’d have to have a university funding structure where faculty can be rapidly hired in proportion to increasing enrolments.

Any ideas on how the models could be refined – or leveraged? This is just a first stab at modelling this. Let me know in the comments.

Reblogging a recent and thought-provoking post by Soraya Chemaly:

<reblog>

“Homophobia: The fear that another man will treat you like you treat women.”
~ (unattributed)</reblog>

By all means, this fear isn’t the entire basis behind homophobia – but it’s a side of it I hadn’t thought of before. The comic that she posted is based loosely on a series of articles by Andrew Sullivan, including this tale:

You struck a nerve with this one [article], as I was just discussing this very thing a few weeks ago with a group of high-school freshmen in my English class. We were discussing homosexuality because of an allusion to it in the book we were reading, and several boys made comments such as, “That’s disgusting.” We got into the debate and eventually a boy admitted that he was terrified/disgusted when he was once sharing a taxi and the other male passenger made a pass at him.
_
__The lightbulb went off. “Oh,” I said. “I get it. See, you are afraid, because for the first time in your life you have found yourself a victim of unwanted sexual advances by someone who has the physical ability to use force against you.” The boy nodded and shuddered visibly._> “But,” I continued. “As a woman, you learn to live with that from the time you are fourteen, and it never stops. We live with that fear every day of our lives. Every man walking through the parking garage the same time you are is either just a harmless stranger or a potential rapist. Every time.”
The girls in the room nodded, agreeing. The boys seemed genuinely shocked.
_
“So think about that the next time you hit on a girl. Maybe, like you in the taxi, she doesn’t actually want you to.” _

Imagine your CS department is a lake.
The fauna of your lake are primarily fish and frogs. Normal lakes in your biome tend to have a food chain where about half of the predators are frogs and the other half are fish.

Back in the 80s, the predators in your lake’s ecosystem used to be 40% frogs. But then the frogs started dying, or leaving, or not hatching, or whatever else caused their population to plummet. Now you’re at 25% frogs. And even though it looks like the fish are doing fantastically, there’s less of them than there should be.

Somebody noticed that the nearly all the fish you see these days are either the white-coloured fish species, or the yellow-coloured fish species – the black fish and the red fish, for instance, are doing about as well as the frogs.

But since the lake is now 75% fish, you’re really more concerned about the frogs. So, you put your efforts into releasing more frogs into the wild. You construct some frog-friendly spaces. You stop by and give them some canned flies to eat.

The problem, though, is that the frogs are still gonna keep dying if the lake is polluted. The frogs are an indicator species – amphibians are more sensitive to environmental toxins. And that pollution is having the same effect on the red and black fish species, if you were paying more attention to them. The two fish species that are less affected happen to have some adaptation that is protecting them.

If you start to remove the pollution from the lake it helps both the fish and the frogs. Sure, it has a larger impact on the frogs, but it also helps the fish. Some species and subspecies of fish were also getting disproportionately hurt by the environmental toxins too, and it’s good for them.

If you want to fix the ecosystem, you have to fix the ecosystem for all the species involved. A culture polluted with the stereotype of computer scientists being asocial nerds in basements hurts the whole environment. It pushes women and other minorities away from going into the field, thinking it’s not where they can belong.

For real change to happen in CS, we need to move past band-aids like Women in CS clubs and mentorship programmes for African-Americans. These initiatives have a role to play in the solution, but they can’t do it alone: We need to get rid of the pollution. Let’s work on cleaning up the lake for our whole ecosystem: it’s not about the frogs, it’s about the lake.

### How Do We Clean Up The Lake?

I’m doing my PhD on diversity in CS. So far in my research, I have come to a hypothesis about diversity initiatives.

Here’s my assertion: _A women in CS initiative is more likely to be successful in the long-term if that _particular_ initiative also helps other minorities (e.g. aboriginals, Blacks, Hispanics, people with disabilities, first-generation university students)._

And by “particular initiative” I don’t mean “we started a mentorship programme for women in CS, somebody could make a mentorship programme for students with disabilities in CS” – I mean “we started a mentorship programme that all students can participate in”.

Why is this important?

First off, well-intentioned diversity initiatives can sadly be counterproductive:

By calling out women for extra help, it reinforces that they are different, which reinforces their otherness. It can also trigger stereotype threat. Overtly feminine role models hurt girls’ interest in science – these women are seen as being unrealistic, and are not mentally categorized as scientists When teaching/recruiting, women speaking about CS are more likely to say the field is “hard” – and perceived difficulty pushes prospective students away from CS Diversity programmes cause people to take discrimination cases less seriously
These types of initiatives are also prone to survivorship bias
The empirical research we have on hand about what has long-term benefits shows us that the tricks that help women help everybody – and unintuitively, many of these are very small changes to make! Using assignments in first-year CS that demonstrate how CS contributes to society
Value-affirming exercises to decrease stereotype threat Mentorship programmes for all students
Exposure to non-traditional but non-counterstereotypic role models Ditching the nerd stereotype when depicting CS in media
Shifting students from fixed mindsets to growth mindsets– for example, by talking about the struggles that great scientists have encountered and how they surpassed them – rather than focusing on their achievements Improving parental leave – having men and women share parenting more equally keeps more women in CS
* Letting students know that it’s normal to worry you won’t fit in at first, but that it will go away with time
Improving diversity for all underrepresented groups makes it easier for the other ones to show up – you’re removing pollution from the lake. Women are more likely to feel at home in a CS department that is multiracial – the more you pry CS away from being a white-and-Asian-boys club, the more you help all underrepresented groups join the community.

We already know a lot about why women drop out or aren’t interested in CS. Let’s target those underlying reasons and fix them for all students, without framing them as women’s issues or ghettoizing women. It’s not about the frogs – it’s about the lake.

Here in Canada, today is the National Day of Remembrance and Action on Violence Against Women, in remembrance of the Montreal Massacre.

24 years ago, a madman went on a mass-shooting at the École Polytechnique of Montreal. He killed fourteen female engineering students whose only “crime” was being a woman. A woman who wanted to learn engineering.

His suicide letter and eyewitness accounts indicate he was motivated to kill these women for being feminists.

24 years ago, being a woman engineering student was an act of social change so radical – so role-incongruent – that a madman like Lépine considered it punishable by death. Being a woman in engineering was an act so radical, so role-incongruent, that it was (and is) inherently feminist.

And as comforting as it may feel that the Montreal Massacre is history – or simply the act of a madman – women in engineering continue to face a much more aggressive sexism than we see in the other sciences. 69% of female engineers have experienced sexual harassment on the job. We still have a long way to go.

Part of the controversy surrounding affirmative action and other systems which give preferential treatment to minority groups comes from the ideal notion that people are judged on their merits – and not their gender/race/etc [6]. In such an ideal world, for instance, a female scientist would be equally likely to be hired, given tenure, or accolades as an identical male scientist.

Science likes to bill itself as a meritocracy, in which scientists are evaluated only their work. A lot of the unease many scientists have about preferential treatment is that it goes against that ideal of meritocratic science [5]. So, it’s worth asking: is a female scientist equally likely to be hired/tenured/etc as an identical male scientist?

Probably the best study design for probing this type of question are correspondence tests. These refer to studies where you describe either a female individual or a male individual to a group of participants – keeping everything but gender (or race, ethnicity, etc) constant – and see if participants respond differently to to the woman/man.

Correspondence tests are generally easier to run than audit studies, where you hire actors to be identical to one another except for gender/race/etc. Both types of studies are useful for identifying discrimination against particular groups. Another approach is to pair real male and female scientists with equal on-paper qualifications and see whether they are equally likely to be given tenure. This approach, however, suffers from the problem of pairing: are that female and male scientist really identical except for on-paper qualifications?

In this post, I’ll be describing the results of three correspondence tests looking at discrimination against women in science. These three studies are also the only such studies that I know of to have been published since the 90s. (There’s an older one from the 70s that is now a bit dated.)

The effect of gender on tenurability

Published in 1999, Steinpreis et al [1] ran a correspondence test looking at the effect of gender on tenure. They sent out hundreds of questionnaires to academic psychologists (randomly selected from the Directory of the American Psychological Association). The paper describes two studies, one on tenurability, and one on hirability. Over a hundred questionnaires were returned on the tenurability study of the paper.

Each questionnaire contained a CV, and participants were asked to rate the CV: would they tenure this individual? Participants were told only that this was part of a study on how CVs are reviewed for tenure decisions.

The CV was that of a real psychologist, who had been given early tenure – a random half of the participants got a version of the CV with the name changed to “Karen Miller” – the other half of the participants got “Brian Miller”. (The questionnaire also asked if the participant recognized any of the names on the CV – such participants were removed from the analysis.)

Each questionnaire had a hidden code on the sheet that indicated the gender and institution of the participant it was sent to – this way the researchers did not have to ask their participants for their gender (which could bias them by getting them consciously thinking about gender).

The results? “Brian Miller” and “Karen Miller” were equally likely to be offered tenure – but participants were also four times more likely to write cautionary comments about “Karen” than “Brian”, such as “I would need to see evidence that she had gotten these grants and publications on her own” and “We would have to see her job talk”.

A caveat of the study, however, is that this was the CV of a psychologist who had been offered early tenure – in short, this was an unambiguously competent applicant. In correspondence studies used in non-science contexts, ambiguity has been repeatedly found to play a large role: a minority applicant who is ambiguously qualified for a job (or loan) is less likely to receive the job than a majority applicant – but in the face of unambiguous qualifications, biases are muted [2].

The effect of gender on hirability for a tenure-track position

The Steinpreis et al paper contains another correspondence test, looking at the effect of gender on applying for tenure-track jobs. They used roughly the same approach for this as they did for the other correspondence test, also with over a hundred participants.

Here, the CV was that of the same individual, but at an earlier stage in her career; the dates were shifted to make it seem recent. Like the other study, participants either saw the CV as that of “Karen Miller” or “Brian Miller”. Participants were asked if they would hire the applicant, and what starting salary they would suggest.

Unlike the tenure study, significant differences were seen between the ratings of “Karen” and “Brian”. “Brian” was significantly more likely to be hired, and he was significantly more likely to be rated as having adequate research experience, along with adequate teaching experience and adequate service experience. He was offered a larger starting salary.

Both female and male participants demonstrated these biases – there was no effect of the participant’s own gender in any part of Steinpreis et al’s two studies.

The effect of gender on hirability for a lab manager position

Thirteen years after the Steinpreis et al study, Moss-Racusin et al ran a correspondence study looking at the effect of gender on the hirability of a canditate for a lab manager position [3]. The candidate here is somebody with a Bachelors degree – this is a lower-position job than the tenure-track job in the Steinpres et al paper.

Moss-Racusin et al sent job packets to 547 tenure-track/tenured faculty in biology, chemistry and physics departments at American R1 institutions, found through the websites of those departments. 127 respondents fully completed the study.

Each job packet contained the resume and references of the applicant – who was randomly assigned either the name “Jennifer” or “John”. Unlike the Steinpres et al study which used a real scientist’s CV, this job packet was created specifically for the study. The applicant was designed to reflect what the authors described as a “slightly ambiguous competence”.

“John”/“Jennifer” was “in the ballpark” for the position but not an obvious star [3]. They had two years of research experience and a journal publication – but a mediocre GPA and mixed references.

Similar to the Steinpreis et al study on hirability, “John” was statistically significantly more likely to be hired than “Jennifer” and was offered a larger starting salary. He was also rated as more competent than “Jennifer”. Participants also indicated a greater inclination to mentor “John” than “Jennifer”. While the differences in the ratings between “John” and “Jennifer” were not huge, the effect sizes were all moderate to large (d = 0.60-0.75).

And similar to the Steinpreis et al study on hirability, both female and male participants were equally likely to rate “John” above “Jennifer”.

The effect of gender on perceived publication quality and collaboration interest

In the two papers I’ve described above, the correspondence studies all used job application materials for the “correspondence”. Knobloch-Westerwick et al [4] took a look at gender discrimination through a different lens: by having participants rate conference abstracts whose authors were rotated as female or male.

Participants in this study were graduate students (n=243), all of whom were in communications programmes; abstracts were all taken from the 2010 annual conference of the International Communication Association. Participants had not attended this conference.

Unlike the Steinpreis et al and Moss-Racusin et al papers, participants evaluated multiple “correspondences” – they each saw 15 abstracts. The participants rated each article on a scale of 0-10 for how interesting, relevant, rigourous, and publishable the abstract was. Participants also rated abstracts on how much they would like to chat to the author, and potentially collaborate with them.

Overall, they found that abstracts with male authors were rated as having statistically significantly higher scientific quality than when these abstracts were presented with female authors. Abstracts with male authors were more likely to be deemed worthy of talking to – and collaborating with – the author. The gender of the participant did not have an effect on the ratings they gave.

Given the other studies I’ve described here, this probably isn’t surprising. What I found quite neat about the paper is they then broke it down by subfield. There’s two steps to this analysis. Before the ran the main study, they ran a preliminary study with assistant professors, which involved these participants rating whether a given abstract fell into a female-typed subfield, or into a male-typed subfield. Knobloch-Westerwick et al then rigged the abstract selection in the main study to show equal numbers of abstracts from these three categories. Female-typed subfields turned out to be communications relating to children, parenting and body image; male-typed subfields were political communication, computer-mediated communication, news, and journalism. Health communication, intercultural communication were rated as gender-neutral.

In female-typed subfields, female authors were rated higher than male authors. In male-typed subfields, the male authors were rated higher than female authors. And in the gender-neutral areas, female and male authors were rated equally. (It should also be noted that the female-typed abstracts were rated less favourably than gender-neutral and male-typed abstracts.)

This brings us to role congruity theory. Role congruity theory looks at gender through the social construct of gender roles – gender roles not only represent beliefs about the attributes of women and men, but also normative expectations about their behaviour [4]. What we saw in the subfield results is that people who are role-incongruous are discriminated against.

Per the article, “Role congruity theory postulates that bias against female scientists originates in differences between a female gender role and the common expectations towards individuals in a scientist role.” [4] In short, where women go against societal gender norms, they’re viewed less favourably.

Discussion

From the articles here, two things emerge for whether we’ll see discrimination against women in a correspondence study: whether they’re role-congruent or role-incongruent, and the level of ambiguity in the study.

When faced with an extraordinary tenure candidate, it doesn’t matter whether they’re role-congruent or role-incongruent. Bias is more likely when the applicant is ambiguously qualified (which is really most of the time). Often, this will mean that a female scientist needs to be more qualified to get the same job: a male scientist can get it for being “good enough” – but the woman needs to be amazing.

The three papers together paint a fairly clear picture that subtle bias occurs against women in science – and that female academics are just as prone to bias as their male colleagues.

While there are a number of issues with affirmative action and other preferential treatment systems for minorities (see [2]) – the notion that scientists are rated independent of gender isn’t one of them. These biases exist and add up quickly over an entire discipline – and over the course of an individual’s life.

References:

1. Steinpreis, R. E., Anders, K. A., & Ritzke, D. (1999). The impact of gender on the review of the curricula vitae of job applicants and tenure candidates: A national empirical study. Sex roles, _41_(7-8), 509-528.
2. Heilman, M. E., Block, C. J., & Stathatos, P. (1997). The affirmative action stigma of incompetence: Effects of performance information ambiguity. Academy of Management Journal, _40_(3), 603-625.
3. Moss-Racusin, C. A., Dovidio, J. F., Brescoll, V. L., Graham, M. J., & Handelsman, J. (2012). Science faculty’s subtle gender biases favor male students. Proceedings of the National Academy of Sciences, 109(41), 16474-16479.
4. Knobloch-Westerwick, S., Glynn, C. J., & Huge, M. (2013). The Matilda Effect in Science Communication: An Experiment on Gender Bias in Publication Quality Perceptions and Collaboration Interest. Science Communication.
5. van den Brink, M., & Benschop, Y. (2012). Gender practices in the construction of academic excellence: Sheep with five legs. Organization, _19_(4), 507-524.
6. Crosby, F., & Clayton, S. (1990). Affirmative action and the issue of expectancies. Journal of Social issues, _46_(2), 61-79.