Category Archives: Common Core

Technology, Tests, and the Art of the Essay

Posted on by
Reply

Should computers grade essays?

The Common Core is trying to shift the emphasis of education toward more complex forms of thinking. Evaluating more complex thinking, however, requires more complex forms of assessment. I think most people would agree that written essays are better indicators of a students’ understanding than multiple choice, fill-in-the-bubble tests, but they are also more time-consuming and expensive to mark. Multiple choice tests can be fed into a computer and instantly graded, whereas essays require a teacher or professor or test center professional to read and evaluate them. Or maybe they don’t.

edtech, assessment, education

Can computers properly assess essays?

Several studies in recent years, like this one for example, have shown that computers can mark essays with the same accuracy and consistency as humans. In fact, computers are often more consistent than human readers. As states struggle to put together new assessments, without breaking budgets, computerized essay grading holds some obvious attraction. Namely, it’s much cheaper.

But it is also controversial and it’s not hard to imagine why. Computers can’t really measure creativity or originality. And the values placed on certain features, like longer words and more complex phrases, open possibilities for manipulating the scoring system.  To see more on this argument, check out this statement from the National Council of Teachers of English (NCTE).

What do you think, dear readers? Do you see any problems with computers grading essays? Leave comments below!

Straight out of the box: Testing ‘cognitively demanding’ tasks

Posted on by
Reply

Last post on these PISA results, I promise. As I mentioned last week. US students are not great at applying math to real world problems. But, according to the OECD’s report, students in the US are good at “reading data directly from tables and diagrams – requiring students only to understand a short text and read single values directly from a representation provided such as a table or bar diagram.”

PISA, OECD, United States

Standardized tests attempt to assess more complicated thinking.

Wow. Seems US students are good at reading charts. They are also good at taking information directly from tables and inserting numbers into formulas already given to them. In other words, the super boring stuff. They are less good at applying any of those numbers and formulas to real world problems; that outside the box type of stuff.

This makes one wonder, or at least makes me wonder, if our strength on easy stuff and difficulty with hard stuff might ironically result from our emphasis on standardized tests. The teach-to-the-test approach seems like it would offer students a whole bunch of opportunities to read from charts and plug stuff into formulas without necessarily connecting any of that to real world situations. In other words, maybe we’ve been going for the low hanging fruit by banging these simple processes into students’ heads.

The US does seem to test more often than other countries. In the US, 80% of students attend schools where student data is posted publically and virtually all students in the US attend schools that track achievement data over time. Those numbers compare with OECD averages of 45% and 72%, respectively. For a country with weak math skills, we do seem to love our data.

Does having all that data mean we are testing more? Maybe not, but probably it does. The report also notes that the US is one of the only countries that relies not only on testing at the national level, but at regional and other levels also.

The Common Core was designed to help us improve our scores. I hope it does. Well, I hope it helps us improve our analytical skills, thus improving our scores. The OECD seems to think it will help and mentions the new standards multiple times throughout the report. A shift from plugging numbers into formulas to more “cognitively demanding” tasks should clearly be a welcome one.

Still, I wonder if any standardized test can really capture the important skills involved in “outside of the box” thinking.  And if not, then how do we determine whether our educational policies are really working? Any opinions on this from our readers?

Another Slice of PISA: Socioeconomic factors weigh heavily on US students’ test scores

Posted on by
Reply

I wrote about the OECD’s annual standardized test earlier this week, the PISA (Programme for International Student Assessment), and today I am going to write about it again. Repetition is the key to these standardized tests, you know. In this particular post, I’d like to take a look at a few of the more subtle details from the report released by the OECD together with the results.

PISA, OECD, socioeconomic status, education

The OECD’s PISA test shows that socioeconomic factors strongly influence educational outcomes.

The first of those details is a big one because it relates to money.  According to the OECD’s report, socioeconomic differences account for a 15% variance in the testing outcomes in the US, compared to less than 10% in Finland, Norway, Japan, and Hong Kong. In other words, two students from different socioeconomic backgrounds are more likely to perform differently on the PISA in the US than in most other OECD countries. And it is not the student on the lower end of the financial spectrum that performs best.

Also, the OECD has a special classification for super rad students that are among the lowest 25% socio-economically but still perform in the top 5% academically. They are called “resilient.” In the US, only 5% of low income students classify as resilient, compared to 7% on average and around 15% for Hong Kong, Macao, Shanghai, and Vietnam.

This seems like it should be a strong point for the US. American dream, social mobility, all that, but as it turns out, it is the opposite. Ironically, unlike many other countries, the US does not show a big difference in the student-teacher ratio or teacher education levels in lower performing schools. So what is that difference? Facilities maybe? After school programs?  I’d be very interested to hear any ideas in that big comment box below.

In terms of school performance, some interesting results came up as well. First, across all the OECD countries, schools with more autonomy tended to perform better. So, the more control the principal had, when combined with accountability measures and strong principal-teacher interaction, the better the students did.

At the same time, there was no cross-country evidence that competition among schools in any way contributed to better student performance. In other words, students at schools that compete to enroll more kids performed at the same level as students that are pretty much stuck with their schools.

On the positive side, or not really, the US did score well in opinions about our own math skills. This comes despite a below average ranking in math. For example, 69% of respondents felt confident in their ability to calculate figures such as the gas consumption rate of a car. The OECD average was 56%. So, at least we are confident. And have warning lights for low fuel.

U. S. Ain’t: The US shows no progress in international education test

Posted on by
Reply

The world is ending. Or at least it is in America. Or at least it is in American schools. After years and years of bold educational reforms, 15 year-olds in the US scored just about where they always have, when compared to students in developed countries around the world: about average in reading and science, lower in mathematics.

US education. PISA, OECD

Despite years of reform, the US continues to perform poorly on the OECD’s PISA test.

Surprising no one, students in East Asian countries knocked it out of the park, and those in Europe did pretty well, too. Meanwhile, among the 34 OECD countries, the US scored 17th in reading, 21st in science, and 26th in math. The margin of error is about three there, for all the optimists and pessimists out there.

The poor performance comes despite spending a ton of money. Among all the countries surveyed, the US had the third highest per capita income, and only Austria, Luxembourg, Norway, and Switzerland spend more per student. To illustrate, the Slovak republic spends around $53,000 per student and performs at the same level as the United States, which spends $115,000. Yikes.

And yea, it gets worse. The US did particularly poorly in measures of critical thinking. Things like, applying mathematics to real world problems. This is supposed to be our strong point. Or at least that’s what I always thought. Like, maybe we drop out of school like Bill Gates, but we still think outside of the box and all that. These test results seem to indicate otherwise.

Of course one might wonder how a standardized test can legitimately measure a student’s ability to think critically. For example, a student with strong critical thinking skills might realize that there are few personal rewards for doing well on these tests.

But then, maybe that attitude is just the problem, a reflection of our American individualist way of looking at life. I am not going to get into all that now, other than to say I always tried my best on standardized tests.  I promise.

So, what next? The OECD says that the Common Core standards should help. We’ve discussed those standards in our blog here and plan to discuss them more. One of the main selling points of Common Core standards is that they emphasize critical thinking. It is tough to prove that one way or the other, especially when standardized tests are actually “the box”, in that “think outside the box” metaphor. But it will be interesting to see.

We will dive into these results a bit more in the next blog post. Meanwhile, don’t lose heart. We still have Bill Gates, Warren Buffet, and Bob Dylan.

The Common Core and Its Many Colors

Posted on by
Reply

Henry Ford once said that any customer could have a car painted any color he liked as long as it was black. Many opponents of the Common Core standards see policymakers giving teachers a similar offer. Teach whatever you want as long as it is the Common Core. Proponents say the standards set broad goals and give freedom to teachers to employ their own methods toward reaching those goals.  Critics say they box teachers in, preventing them from adjusting their class material to the needs of their students.

Common Core

How much freedom does Common Core give for teachers to teach?

The basic idea behind the Common Core is that students around the country should study similar topics at similar times.  For example, all students should study subtraction in the second grade and past tense verbs in fourth.—Disclaimer: I am making these up for the sake of illustration. I am already too far behind schedule to read through all the standards. Anyway, the point is, all the teachers have the same goals and can use various methods to reach those goals.

Proponents of the standards see them as key to equalizing educational experiences and educational opportunities around the country.  It doesn’t seem fair that students in rural Kentucky wouldn’t get opportunities to study the same material as the students in New York City. Or vice versa. Common Core material also prepares students for SAT and ACT college entrance exams. By making sure all students cover the same material, Common Core proponents are hoping to level the playing field.

That approach may ignore circumstances, however, which require teachers to slow down and make sure students understand the material. Students in low income areas, for example, or whose first language is not English, may not be ready to study the same topics as students in an upper class suburban neighborhood.

Further complicating things, the combination of the Common Core and the No Child Left Behind policy produces a whole lot of tests. Those test results significantly influence a teacher’s future. Weighing those test results too heavily would reward a teacher for effectively teaching test-taking skills, but not necessarily being good teachers. In this sense, like Henry Ford’s offer, teachers can teach anything they want as long as it is the stuff on the tests.

Let me throw out a hypothetical here: What if a teacher has to spend extra time helping students learn to play well together? Seriously, I’m not joking. Students from rough backgrounds often need time to learn to get along better with each other. That is a valuable, life-long skill. And hats off to a teacher that can actually teach that. But will it help on those dang test scores? Probably not. And if so, maybe not until a few more years down the road.

I mean, surely it is not bad to have some general direction on what to teach in class and occasional measures of effectiveness in teaching that material. The problem is, it is tough to properly measure all the many roles that teachers perform. The most profound ways in which teachers influenced me were only loosely related to coursework. So how can we measure the importance of a teacher that inspires children to travel, to help others, to be good friends and neighbors?

I don’t have the answer for that, but I am very interested to know what our readers think. How can you judge the effectiveness of teachers when they are expected to perform so many different roles?  And how much flexibility should teachers have in determining the academic material covered in class?