How to use AI to run activation tests

Perplexity has a browser - Comet.

I think it's a fabulous tool to run activation tests and to friction log.

The premise is that if AI runs into issues using your app, chances are that a human would do too.

Let's try to find friction for three companies:

• MakeMyTrip (similar to Booking.com)

• Canva

• Calendly

The first step is to identify a core action. In the case of MakeMyTrip, we want to search for flight tickets.

Watch the recording below and try to identify some obvious issues.

One issue that you might notice is that it books a return ticket automatically. The default return ticket falls outside MakeMyTrip's own edge cases and throws an error.

The website also takes a long time to load.

Moving on to another test.. Canva.

Canva runs a lot of user tests using usertesting.com.

But Comet still ran into a few issues, that I believe humans could run into as well.

1. Canva uses a little crown icon to distinguish free vs pro templates. Comet didn't quite understand and ran into paywalls thrice in a row.

2. It accidentally exported a PNG when it should've exported a GIF or an MP4 format. It missed the little alert Canva showed.

3. It had difficulty closing the templates sidebar to pick elements.

4. and so on...

Some of them might be in this video below:

Calendly had the best UX of all the activation tests I ran.

AI could smoothly complete the journey.

The only two places I saw it get stuck was the timezone selector and the extra popups.

1. The timezone selector wasn't obvious for the AI to select, and the default one from Calendly was incorrect.

2. It also had to close an extra popup which I thought could be removed.

IRCTC (railway site) was the hardest of them all. AI just couldn't get through to any possible action.

It didn't understand why the "Book Now" button was greyed out after a search was made.

The point of this blog is not to identify issues at Canva/Calendly.

The point is to show that AI has the ability to find issues that might not be obvious to us.

You could run it to do mini feature tests or full blown ICP tests.

You can chat with in real time, pretending to be a real user. It also takes screenshots in real time, so you don't have to.

Download it here: perplexity.ai/comet