One day, my big ass VW Touareg burst into flames on my driveway. That’s a real picture of the incident at the top of the page. What a morning that was. Anyway, as you can imagine, I needed to set about buying a replacement car soon after. I chose a Nissan Leaf (I’m practicing for a Tesla) and I have to say I love everything about it, almost without exception.
But there is an exception. The battery on the car is only enough to drive about 60-80 miles on a charge which means I need to top up nightly for my commute. Which should be fine, but on occasion I forget to plug it in. Not often, maybe once every two months; something distracts me when I’m getting out of the car and I forget to plug it in. Worse still, I only find out the following morning when I’m about to drive to work and can’t.
By the third time this happened, I decided enough was enough and I needed a technological solution and ‘leafcheck’ was born. The premise was simple: every day at 8pm, I wanted an automated service to check the charge state of my car (using it’s API) and send me a mail or SMS if it wasn’t plugged in.
During the recent holiday period I decided to build my solution – a simple NodeJS app running in a docker container on an Azure VM would do exactly this. And it works very well. Except for the one night when I popped to the store at 9pm and forgot to plug it in after the scheduled check had taken place. That’s when I realized I needed an app that used a geo-fence to know when I’ve arrived home, and checks the charge status 5 minutes later.
I built this just for me, just for fun. But a friend thought it would make a pretty interesting business idea – “why not create a subscription service that other leaf users can sign up to, and if it’s successful you can extend to other types of EV”. I argued that only I was absent-minded enough to keep forgetting and there was no market for this. With about 50,000 LEAF users in the US, a 1% adoption rate (pretty good) would yield just 500 folks – so I wouldn’t be quitting my job any time soon. But this could be a bit of fun and I was curious to test the idea using lean techniques and so www.leafcheck.com was born.
This was my MVP (minimum viable product) to prove my hypothesis:
“Only I am absent-minded enough to spend time or money on a service that shouts at me when I forget to charge my car”
The website really was minimal:
- Some simple content in a bootstrap template
- A form, the first stage of the subscription process
- Google analytics
- Optimizely for some A/B tests
The whole thing took about 1-2 days effort, including my struggles with the bootstrap template. And whilst the site suggested there was a ‘checkout process’, there wasn’t. The first page of the subscription process collected user details like name and e-mail and notified me. I was really seeking to measure the intent to subscribe – I didn’t have something of service quality to sell yet.
Ideally I would have asked for Credit Card numbers – as that’s the ultimate indicator that a customer is interested in subscribing but I wanted to avoid any complexity dealing with Credit Cards might pose. So my checkout process just ended abruptly with a “thanks, we’ll be in touch”.
On pricing
Pricing is tricky – something is worth what people are willing to pay for it. How much is this service worth? How much would I pay for it?
I decided to double down on the experimentation and use A/B testing to experiment with price too. Using Optimizely, I ran an A/B test that adjusted the advertised price. Users would see either:
A: $1.99 per month / $15.99 per year
B: $2.99 per month / $24.99 per year
C: $3.99 per month / $46.99 per year
Optimizely really is an incredible tool. I can view my webpage in their browser based editor and change the text or HTML on the fly for each variation.
Below, you can see how I even edited the href links for each variation so I can track it through my funnel (p=low below means variation 3, or lowest price).
In theory, with enough traffic, I’d be able to estimate the best price to go with based on the percentage of conversions. Of course, this assumes my hypothesis was wrong and somebody else is crazy enough to pay for a service like this. It’s all about the learning though, and so onward…
Driving Traffic
Building a site and driving traffic is hard. Building a site that can drive organic search takes a very long time and I wanted this experiment to be short and provide rapid feedback. Fortunately, social networks like Twitter and Facebook make it very easy to target a specific type of user. For example, on Twitter I can promote tweets to people who follow @NissanLeaf and likewise with facebook, people who ‘like’ Nissan Leaf. Surely a fantastic way to find other leaf owners! I might be even more targeted and go after people who follow @NissanLeaf and @ICanImproveYourMemory but the audience overlap was too narrow.
And with a shoestring budget (no more than $100) I launched my campaigns.
Facebook Results
It’s amazing how easy facebook make it to advertise. Their experience is slick. As I entered my URL they automatically harvested my website to design ads using images from the site. I was seriously impressed with how easy they made it for anybody to get a campaign up and running. It took me less than 20 minutes to be up and running.
These are the folks that saw the ad.
Nearly 10,000 in total. 110 folks clicked for a CTR of just over 1%, which is pretty good[1].
Of the 110 folks that made it through to the website – 12 clicked on the buttons to start the subscription process: 4 at the top price, 4 at the low price and 5 at the middle price.
[1] Clearly, the numbers in this article aren’t large enough to support statistical conclusions – but interesting to see what you might be able to do with larger numbers and a larger budget. This was for just a $50 spend.
None. Nada. Nobody actually converted and submitted their details. Not enough data to prove anything, but zero conversions from 110 clicks isn’t encouraging. On to twitter…
Twitter results
Twitter also do a pretty good job of making it easy to advertise – although I had to wait a while for my new @leafcheck account to qualify for advertising. Like facebook, I invested 50 hard earned bucks on my twitter ads, here’s the results:
My $50 earned me just 2,642 impressions in total, with an average CTR of 0.95%. As you can see, the lower ad with the image had a much higher CTR with 1.03% vs 0.51%.
So in this case my $50 earned me 25 clicks – just one quarter of those I received from facebook. This isn’t to say that facebook > twitter for advertising – I’m sure there are many variables that decide what’s best for each problem. Here’s the funnel for each:
Note that the lower cells look at the percentage of the previous stage acquired, e.g. of all facebook impressions – 1.1% clicked. Of those that clicked, 10% went through to sign up. Nobody actually signed up.
I think we can assume that my hypothesis: “Only I am absent-minded enough to spend time or money on a service to shout at me when I forget to charge my car” was proven and I declare the fast failure of project leafcheck a success. Or depending on your point of view – a fail – and the image of a burning car for the post is ironically apt. But I had fun in the process and probably saved a lot of time, had somebody managed to convince me this was a viable project.
Of course, It is possible (though unlikely) that this is a viable project. Conversion rates for most premises are in the very low percentage numbers, so it’s likely I’d have to have driven much more traffic to the site before I see a conversion – that’s just the cost of customer acquisition. Maybe with simple modifications of the text and subtle changes to the site I could have increased conversions significantly, but I’d need to drive a lot more traffic to support reasonable statistical analysis and to support re-testing future revisions – which would cost me money I’m not willing to spend on my little pet project.
Don’t let good data go to waste
There’s no doubt that leveraging data and using it to drive decisions is an incredibly useful technique – if you’re lucky enough to have data pumping through your site/product/system/app make sure you’re capturing and leveraging it. It’s expensive to buy if you don’t.
introspective.io 