Dear Google, let’s talk about webapps.

screenshotI threw together a little hack project last weekend, called Subwalkway. It’s mobile-only, and it’s a bit of a mess- part UI experiment, part subway navigation tool, and rough around the edges. But if you’re on an iPhone, try adding it to your home screen. It looks right, doesn’t it? An icon (with an easter egg!) that seamlessly blends with the phone interface, a splash screen when you launch it, and no navigation chrome when you’re using it. If you squint a little,  you could almost imagine that it was a native app.

Now try doing that on an Android phone. Actually, don’t bother, I’ll save you the effort- it does precisely none of these things. If you’re using Google’s Chrome browser you can’t even add a site to your home screen*. So, today I ask: Google, what the hell? From email to entire office suites, you’ve spent years trying to convince us that the web is the future for software- you even went as far as to create an entire OS based around it. We could be making responsive webapps that work great on ChromeOS and adapt to being perfect first-class citizens in Android- if you let us. Instead we’re forced to provide sub par in-browser experiences, or wrap our apps up in a clunky WebView frame and lose all of the performance and automatic updating that HTML5 can provide.

Screen Shot 2013-05-10 at 10.34.07 AMOther phone manufacturers are catching up to the game- even though WebOS is dearly departed, Blackberry 10 lets us make HTML-based apps. Firefox OS makes them a first class citizen. Even Microsoft allows you to write apps in JavaScript. Why has it been left to Apple- who have every incentive to trap me in the sickly embrace of their Objective C App Store- to be the pioneer in webapps?

I want to make cross platform apps using HTML technologies, and I want to make them great. How can it be that Google is the one standing in the way of me achieving that?

(Hat tip to Peter Nixey, whose blog post title I shamelessly ripped off)

* As pointed out in the comments, it actually is possible. But it’s little wonder that I never discovered it, given the steps required.

How I served 100k users without breaking the server- or a dollar bill.

Or, “The incredible, affordable S3”

Screen Shot 2013-04-26 at 11.41.51 AMI made a silly thing. The Associated Press’s Twitter account had just been hacked and sent out a fake tweet about an explosion in the White House. The Dow Jones immediately dropped 150 points. So I made a silly thing. It’s called “Is My Twitter Password Secure“, and the best description I could probably give it is a PSA on phishing sites. Try it, you might laugh. If you’re like many people, you’ll be so convinced by it that you’ll send me abuse on Twitter. That’s OK.

But anyway, it “went viral”. Tweet after tweet arrived in my “mentions” box as people enjoyed the joke and shared it with their friends. Despite being hammered with requests, the server never slowed and never crashed. Because there was no server. At least, not that I had to worry about, because I hosted the entire thing on Amazon S3. Not only is it super-reliable, it’s also super cheap- serving 758,509 requests cost me… thirty cents.

Screen Shot 2013-04-26 at 11.41.58 AMOf course, you can’t run any dynamic scripting on S3, but you’d be surprised the number of times you don’t have to. For example, the promo site for my taxi app does use dynamic content- the map tile images are generated by an EC2 instance I have running TileStream– but the vast majority of the page runs quite happily on S3. JSONP or CORS mean that you can quite effectively run an ‘API’ server on an EC2 instance, while leaving the majority of your static HTML on an S3 bucket.

While the steps to set this up aren’t complicated, I thought it might be worth creating The Definitive Guide To Hosting A Web Site On S3.

The Definitive Guide To Hosting A Web Site On S3

The steps are actually really quite simple- make a bucket, set up a CNAME, upload your files. But let’s go one by one:

Make a bucket

Every domain name you want to use has to be a different bucket. Make the name of the bucket the exact domain name you wish to use (including the subdomain, like www.):

Screen Shot 2013-04-26 at 11.25.36 AM

Then you need to make this bucket publicly browsable. Select your bucket, and open up the Properties tab. Under ‘static hosting’, you just need to check “Enable website hosting”:

Screen Shot 2013-04-26 at 11.29.21 AMYou’ll notice that there is also a box for ‘index document’ – you ought to be fine to leave this as it is (no point trying to host PHP files on here, folks) but if you’re one of those people you might need to change it to “index.htm”.

Ta-da! You now have a static web site up and running.

Upload your files

Unfortunately, S3 doesn’t offer anything so simple as FTP access. That said, there are many clients out there set up for S3 uploads- my personal favourite is Cyberduck, it’s donationware, and supports just about every uploading scenario you could wish for. All it needs is your AWS API key and it’ll list all your buckets out for you, and let you drag and drop your files straight into your bucket.

Get a better domain name

That endpoint URL is pretty gross. Thankfully, it’s very simple to get a better one mapped to your S3 site. I’m using Namecheap in these screenshots, but any DNS provider ought to be able to do the same thing. Go to edit your DNS records, and add a CNAME record for your chosen subdomain that points to your gross endpoint URL:

Screen Shot 2013-04-26 at 11.37.51 AM

That ‘IP ADDRESS/ URL’ field’s full value is – that last full stop on the end is important. And the URL redirect above simply directs all users to the www subdomain if they haven’t already entered it.

And that’s it. Your static site is up and running on your pretty domain name. Now you’re free to play horrible tricks on the world without worrying that they’ll crush your server and/or wallet in return. Use this power wisely.

So, you’re going to do the StartupBus.

Congratulations, you are one of the least rational people I know. That’s a good thing. But you need to prepare yourself for what’s coming- it’s an amazing experience and you get to celebrate at SXSW afterwards, but before that comes the most intense bus journey of your life. I should know- I did it last year, and I thought I’d pass on what little advice I have for those who set off in a few days.

You will be stressed. Your teammates will be stressed. You will argue. That’s OK.

After you get into teams, you’ll have a glorious three hours or so where you’re unstoppable. Then you’ll start getting down to details, and people will have different ideas about what direction to take. You’ll have a healthy debate about it. A few hours later, you’ll be sleep deprived and suddenly the smallest question will become an existential debate and you’re right damnit, why won’t everyone else realise that? If it’s 3am and you’re yelling in each other faces about what colour gradient to use in your logo, go to bed. Even if you come to a final decision you’ll probably all wake up the next morning and think it’s wrong.

The important part isn’t trying to avoid arguing (because it will happen), it’s waking up the next morning, forgiving each other for being such assholes and just getting on with it. It’s no coincidence that one of the most successful teams on the NYC bus last year was the one making Happstr.

The internet connection is going to be terrible. No, worse than that.

I was told before I left that the internet would be patchy. “Ah, it’ll be fine”, I thought to my idiot self as I pitched an idea based around streaming music from the cloud. It was a huge, huge mistake- even when we did have a working connection, it could take up to thirty seconds just to start buffering a song. When we didn’t, well, I couldn’t do a thing. So, a few developer-specific tips:

  • Don’t work on an idea that needs to stream lots of data. If you do (don’t), then make sure you can complete the entire process locally. Don’t use, say, the Spotify, Rdio or Youtube APIs, to pick examples at random.
  • Whenever you do use an external API, download sample responses for every call you make, and ideally give yourself a switch between live data and locally cached stuff.
  • Download as much documentation as you can. I was extremely glad to have offline copies of both the jQuery and jQuery Mobile documentation, for example. If in doubt just clone entire GitHub repos at random. You’ve got the disk space.

Be sociable.

Yes, the StartupBus is a competition and you should be very focused on the project your team is creating. But the chances are that the real, long-term benefit you’re going to get from this trip is the people you’re going to meet and the connections you’ll make. So, you know, hang out. If it’s anything like last year, you’ll have numerous chances to meet people from other teams and other buses as you make your way to Austin. Take advantage of that. And although you’re competitive, the other teams aren’t your enemy. Talk to each other about your ideas and get opinions.

Then, when you’ve all arrived in Austin and the competition is over, you can all sit around and laugh about it, as if it was a weird blurry, sleep deprived dream. Because it kind of will be.

Users are blaming Google for the lack of Contacts integration in the new Maps App

“It looks nice but it doesn’t connect with Contacts. That’s a huge flaw. I’m surprised that was overlooked.”

“Does Google Maps not work with Contacts anymore?! If so, this is a MAJOR shortcoming that should be mentioned in reviews.”

“I really like Google maps and the interface on the iPhone, but as several people have pointed out in their reviews on iTunes Store, there is no native link to contacts”

This is a selection of the “most recommended” comments on the New York Times’ article about the new iOS Google Maps app. What does it tell us? That outside of tech circles, people care about OS-level integration (even if they don’t know it), and many people blame Google for not implementing it.

This is a problem. From Apple’s perspective it isn’t- Google gets blamed for a sub-par product, and Apple’s offering retains exclusivity over a core feature. But how long can this go on? It’s not too difficult to imagine that a future version of the Chrome app will intercept map links and forward the user to Google Maps. How long before we get a Google Contacts app that serves no purpose other than forwarding web and maps links to Google products? Would Apple even approve such an app? If so, when can I expect my Yahoo Contacts app to sit alongside my Bing Contacts app in my folder of “useless apps I am forced to use if I want to use other apps I like”?

The solution seems simple- integrate Android-like app preferences into iOS. This isn’t as bad as people make it out to be for one main reason- you won’t see this screen unless you have installed two map apps. If you never download Google Maps, you’ll never be asked to choose Apple Maps. However, if you have, it stands to reason that you might be interested in using it through other methods than from the home screen.

But why would Apple ever add such a feature? They have no motivation to, while users are blaming app developers for a lack of OS integration- they get none of the blame and all of the benefit. But Google Maps might finally be the straw to break the camel’s back- it only took seven hours for it to become the most popular free app in the App Store. The question remains: how can we make it clear to users that Apple is the barrier to their app integration happiness, and not perceived laziness on the part of developers?

Improving <canvas> performance – never underestimate copy and paste

As part of a current pet project (more on that some other time) I stumbled upon a fantastic canvas-based library for creating heat maps- the aptly named heatmap.js. It only has one flaw- when you start adding a lot of data, it’s slow. That’s a problem for me, because I want to animate my heat maps. With no modifications, the map was rendering at around 3fps on my 2010 Macbook Air.

How does this thing work anyway?

The first step was to find out how the library worked in the first place. It’s pretty ingenious- drawing monochromatic blurred circles for each of the data points, at different alpha levels depending on value. Then it redraws pixel by pixel, colorising according to the alpha value of the pixel. This means that overlapping points combine alphas, dictating the overall colour.

Problem #1: this involves using <canvas>’s getImageData() function- which has some serious performance problems. Surprise #1: while different browsers have huge performance differences, all benefitted by splitting the map into multiple canvases and reading them in turn. As it happens, that suits my needs perfectly- I am going to be displaying the heat map on top of tiled map images.

Still not good enough

It was an improvement, but I was still looking at around 12fps in Chrome- not bad, but not ideal. With the getImageData() parts optimised as best I could manage, I looked at the next big drain- the initial drawing of blurred data points. Problem #2: what can I do that is any simpler than drawing a circle? Surprise #2: I can just copy and paste the same circle over and over.

The existing code was drawing it’s circles like so:

I extracted that code and drew it (with 1.0 opacity) on another separate <canvas>. Then all I needed to do for each point was:

It didn’t seem like it ought to be faster- switching between two different tags and running drawImage() seemed like it would be far more complex than drawing a simple circle. But I was wrong- the performance increase was massive- take a look at the before and after test pages. There are undoubtedly further improvements to be made, but my heat maps are now flowing along at over 30fps, which gives me a good amount of leeway as I continue to develop my project. I would love to be able to transfer these calculations to a web worker, but alas they have no DOM, and so there’s no way for me to get a 2D drawing context.

Did I miss something fundamental? Any ideas for improvements? Let me know!

Fun with analytics: pitting Hacker News and /r/programming against each other

I noticed a weird anomaly in my analytics today. It started shortly after midday EST on Monday, and lasted for approximately 24 hours. What was the shape of this crazy UFO-style disturbance? Something like this:

I call it the snapped hockey stick.

The anomaly was my post about making custom maps for my app, Taxonomy. I submitted it to both Hacker News and /r/programming, and to my surprise it was a hit in both. So, delving through the figures, I thought we could have a little fun, and compare Hacker News to /r/programming- the figures, the discussions, and which of you are nice and voted for my app. Continue reading

I had no idea how to make custom maps, so I learnt by doing. You should too.

Lately, I’ve been making an app called Taxonomy. It’s designed to make your taxi experience better- a traveling companion, if you will. Going by the logic that a lot of people (in New York, at least) only really travel in taxis when out at night, I decided that it ought to have a dark colour scheme. Not morbid dark, but, y’know, “mischievous things happen at night” dark. I mostly succeeded- styling my logo, headers and buttons in a dark monochrome with some bright yellow highlights (it is a taxi app, after all), but every time I wanted to show the user’s current location this ridiculously bright Google Map showed up and ruined my carefully cultivated style. Something had to be done.

I played around with the styling options Google Maps now provides. It’s powerful, but it still wasn’t enough. The fonts were wrong, the options were difficult to tweak, and- as I was quickly discovering- the Google Maps API doesn’t even perform that well on mobile devices. I remembered back to a Hacker News post I’d seen some weeks earlier about TileMill, and the beautiful maps I’d seen created with it. Now, I’m just a web developer- I’ve done enough projects to know what latitude and longitude are, but that’s about the extent of my knowledge. Despite that, I decided to dive in and give it a try. How hard could it be? Continue reading