Comparing Online and AI-Assisted Learning: A Student’s View

An 8th grader reviews traditional Khan Academy and its AI-powered tutor, Khanmigo

Illustration

Hi everyone, I’m Daphne, a 13-year-old going into 8th grade.

I’m writing to compare “regular” Khan Academy (no AI) to Khanmigo (powered by GPT4), using three of my own made-up criteria.

They are: efficiency, effectiveness, and enjoyability. Efficiency is how fast I am able to cover a math topic and get basic understanding. Effectiveness is my quality of understanding—the difference between basic and advanced understanding. And the final one—most important to kids and maybe least important to adults who make kids learn math—is enjoyability.

Reviewing Khan Academy

Let’s start with “regular” Khan Academy to see how it fits these categories. Khan Academy is a mix of practice problem sets and video explainers. Just for personal context, I meet with a math tutor once a week, and some weeks I get assigned one or two 30-minute problem sets as homework. I’ve probably worked through more than 50 of these in the past couple years.

In terms of efficiency, Khan Academy is really solid: 5.5 out of 10. If you understand the topic, like adding fractions, then you can do these in a breeze, and the extra practice enhances your comprehension of the topic. However, if you get confused, then what? You can click on hint. But doing that comes at a cost: you can’t get “credit” anymore for getting a right answer on your own.

If you do click on a wrong answer, you get a gray dot that symbolizes wrong. That bothers me. If you’re a perfectionist like myself, it’s possible to spend 30 minutes over that one question you’re unsure of, not wanting to see the gray dot. After a wrong answer, you can click the get help button and receive a lengthy explanation on how you should’ve done the problem. You can then move on and apply the knowledge you learned to the next question.

Okay, now comes effectiveness. Again, my rating is 5.5 out of 10. A regular Khan exercise isn’t very good at teaching you new things—the hint and explanation are not enough if you’re really confused—but is excellent at reinforcing what you know. Now, it may be unfair for me to say this; I realize there are videos to teach a new topic. Me and my friends find them boring—perhaps effective, but I never use them. On Commonsense Media reviews, it seems like lots of kids feel this way.

Last but, in my mind, certainly not least, enjoyability: 6 out of 10. Just all around, Khan feels the same as a normal problem set in school. Let’s be honest, speaking for the majority of kids, math is a chore. (Sorry, Dad.) But Khan tends to be pretty straightforward at helping you get this chore done, and for kids like me that just want to get the work over with, Khan isn’t that bad. Not too special and perhaps a bit bland, but not terrible.

All in all, Khan Academy “Original Recipe” gets a solid rating of 6 out of 10.

Interacting with Khanmigo

Moving on, let’s look at Khanmigo. My dad gave me this instruction: try to learn a topic that I do not know. So I chose scientific notation.

First, let me show you what Migo is. Khanmigo asked me:

Migo types:

What is the scientific notation for the number 3000?

Daphne types:

3^1000

Migo types:

Here’s my take: Scientific notation is a way of expressing numbers that are too big or too small to be conveniently written in decimal form. It’s written as a product of a number between 1 and 10 and a power of 10.

So, for the number 3000, we can break it down like this:

3000 = 3 * 1000

We know that 1000 is 10 to the power of 3 (because 101010 = 1000). So we can rewrite 1000 as 10^3.

So, 3000 in scientific notation is 3 * 10^3.

Comparing this with your answer, it looks like there was a bit of a mix-up. You wrote 3^1000, which means 3 raised to the power of 1000, a number much larger than 3000. That’s okay, though! Mistakes are part of learning.

Now, let’s move to the next question.

Question 2 of 10:

What is the scientific notation for the number 0.004?

Remember, you can ask for a hint if you need one!

Daphne types:

that was to many parts can u go over that again with less words

Migo types:

Sure, I can simplify it!

For the number 3000:
• Break it into 3 and 1000.
• Write 1000 as 10^3 (because 101010 = 1000).
• So, 3000 = 3 * 10^3 in scientific notation.

Now, for the next question:

What is 0.004 in scientific notation?

Remember, you can ask for a hint if you need one!

Okay, reader, Daphne here again. What do you think happened next? Was I able to get the next question right? Let’s find out.

Daphne types:

i dont know how to do this.

Migo types:

No problem at all! Here’s a hint:

When a number is less than 1 and we want to write it in scientific notation, we use negative exponents.

For example, 0.1 can be written as 1 * 10^-1.

Now, try to write 0.004 in scientific notation.

Daphne types:

4 * 10^-0.01

Okay, reader. Imagine you were my human tutor. What would you say at this point if you saw my answer above?

Well, here is Migo’s response.

Migo types:

Here’s how I would do it:

When we have a number like 0.004, we want to move the decimal point to the right until we have a number between 1 and 10. In this case, we move the decimal point 3 places to the right to get 4.

Each time we move the decimal point to the right, we subtract 1 from the exponent of 10. So, since we moved the decimal point 3 places, our exponent is -3.

So, 0.004 in scientific notation is 4 * 10^-3.

Comparing this with your answer, it looks like there was a mix-up with the exponent. You wrote 10^-0.01, but it should be 10^-3. That’s okay, though! Mistakes are part of learning.

Now, let’s move to the next question.

Question 3 of 10:

What is the scientific notation for the number 70000?

Remember, you can ask for a hint if you need one!

Daphne types:

again, too many words. act as if im not very smart

Migo types:

I’m sure you’re smart! Let’s break it down.

For 0.004:
• Move decimal 3 places right to get 4.
• This gives us 4 * 10^-3.

Now, for 70000 in scientific notation:
• Break it into ? and 10000
• Write 10000 as 10^?.
• So, 70000 = ? * 10^?.

Can you fill in the “?” to write 70000 in scientific notation?

Daphne types:

7

Migo types:

Here’s my take:

You’re right about the first part! The number we’re working with is 7.

For the number 70000:
• We break it into 7 and 10000.
• We write 10000 as 10^4 (because 101010*10 = 10000).
• So, 70000 = 7 * 10^4 in scientific notation.

Comparing this with your answer, you got the first part right! The number we’re working with is indeed 7. But remember, we also need to express the number of zeros as a power of 10. In this case, it’s 10^4.

You’re doing great! Let’s move on to the next question.

Question 4 of 10:

What is the scientific notation for the number 0.0008?

Remember, it’s okay to ask for a hint if you need one!

Daphne types:

8 * 10^-4

Migo types:

Here’s how I would do it:

When we have a number like 0.0008, we want to move the decimal point to the right until we have a number between 1 and 10. In this case, we move the decimal point 4 places to the right to get 8.

Each time we move the decimal point to the right, we subtract 1 from the exponent of 10. So, since we moved the decimal point 4 places, our exponent is -4.

So, 0.0008 in scientific notation is 8 * 10^-4.

Comparing this with your answer, it looks like you got it exactly right! Great job.

Reviewing Khanmigo

Okay, reader, Daphne here again. Now here are my ratings of Migo.

For efficiency, Khanmigo is above Khan Regular. 6.5 out of 10 (versus 5.5 out of 10).

If the bot’s response is too long, you can adjust it. (Just ask, “Can you simplify that for me?”) If the problem is too easy, the bot can make the next one harder. You can actually learn a lot more in a lot less time. The only part that bothered me was that sometimes it took a while for the responses to load, so I was sitting there bored waiting for a bit.

For effectiveness, not only can you learn a lot more in a lot less time, but you can learn a lot more in depth. It’s quite different from regular Khan in the that Khan Original is like the work a tutor would assign you, and Migo is like an actual tutor. You can ask questions, ranging from wanting more or less knowledge. So, for effectiveness: 7.5 out of 10 (versus 5.5 out of 10).

And as for enjoyability, I’d say kids’ answers would differ quite a bit. Some would enjoy the engaging, thought-provoking parts, while others would rather just do work independently. Personally, I think I actually prefer regular Khan, which surprises me, but I guess in this rare scenario, I actually like the simplicity of regular Khan. Maybe I just didn’t feel like thinking that hard. So, for enjoyability: 5 out of 10 for me.

All in all, Khanmigo gets an 8 out of 10 holistic rating. Because you learn a lot, quickly.

Conclusion

To conclude, Migo is like a teacher, and regular Khan Problem Sets are like homework. So obviously you learn more from the teacher, but occasionally the teacher can ramble on. But homework that serves merely as practice for topics you already learned in class is admittedly a decent tool with its own perks, even if you might be learning less and the quality is lower.

Daphne Goldstein is an 8th grader at Watertown (Mass.) Middle School. She likes volleyball, psychology, and iced chai lattes.

 

Last Updated

NEWSLETTER

Notify Me When Education Next

Posts a Big Story

Program on Education Policy and Governance
Harvard Kennedy School
79 JFK Street, Cambridge, MA 02138
Phone (617) 496-5488
Fax (617) 496-4428
Email Education_Next@hks.harvard.edu

For subscription service to the printed journal
Phone (617) 496-5488
Email subscriptions@educationnext.org

Copyright © 2024 President & Fellows of Harvard College