Scargill

I recently attended a cracking talk at Teched Developers 2007 in Barcelona… and this article is superficially on the subject of PHOTOS – and what we (by that I mean the world) can do with them – I think you’ll find this very interesting, mind-boggling in fact, but you’ll have to bear with me as I take it through in stages…

Back in the last century, people would go off on holiday, take pictures, take their films along to Boots, get the pictures printed, look at them and then shove them away in a draw to go moldy.
Then along came digital cameras – and so now people take their digital camera on holiday, take WAY more pictures than ever before… and put them on a disk to go, erm, moldy. I’ve been taking pictures since the first digital cameras came out – all neatly categorized and stored in directories which no-one will ever see.

At least I used to. It’s fairly obvious for all to see, the way the younger generation don’t have the same security-centric view of the world that previous generations seem to have – far from it – via the web, they want to show off who they are – and what they’re doing, what they like, what their aspirations are – and so you see sites like Facebook and Myspace popping up – and blogs coming from the least expected places… in other words, significant numbers of ordinary folk, not techies – are putting their lives on the web – or at least the parts they want people to see. Most savvy teenagers in the UK have a presence in the likes of MYSPACE – and put their photos up there, what they’re doing etc… the figures for anyone who’s been down a mine for the past 5 years will take your breath away… more and more people discuss and meet on the web… but that’s not news.

For some time now, I’ve joined the party and now put all of my personal photos on FLICKR, well, the nice ones – the ones in focus etc anyway – and if anyone has ANY doubts as to the number of people who put their photos on Flickr instead of hiding them away, I just looked – and in one minute, try 3,500 photos – That’s not a special minute – that’s EVERY minute. That’s… A LOT (a quick calculation says 2 billion a year). Go to FLICKR and key in just about anything you like and someone’s taken a photo of it. Flickr is BIG – but it’s not the only place online for storing photos by any means.

Here’s an example Flickr search – “Eiffel tower” – 147,000 photos. Even Big Ben managed 82,000 photos.

SO, you can organize your photos, tag them, organize into groups, show them off to friends if you have any… lovely… but so what? Fine if you have friends who live a long way away… but it’s not THAT different to printing them out and showing them off, or is it?

For some time, developers have been doing something called MASHUPS – indeed there are tools to let the public do that – you take something from this site and that site – and incorporate it into your own. Embedding Google maps into a website is commonplace and I’ve now started embedding my FLICKR images into my personal website – and so you get all sorts of nice slideshows you might not want to develop yourself – for free.

Earlier this week I discovered a new tool to let you take images from FLICKR and just about anything else – into an intermediate block – let’s say a book with pages that turn – with your pictures in it – and put THAT in your website. That’s a MASHUP. Ask me about it if you’re interested.

Along comes Microsoft Labs… these chaps been working for some time on something called SEADRAGON.

Stay with me – I’m making a point and it’s worth it.

So – Consider Google Earth – you can zoom in and out to any part of the world and go in with incredible detail….. yes… turned geography lessons upside down I’d imagine – I could not imagine NOT teaching a kid how to use this tool. Of course, you don’t hold data for the whole world in your PC – no, it’s pulled in from the web only as and when needed – as you zoom in from the entire world view which has very little detail– to someone’s back yard! And this is the key to the next stage…

Now consider a screen with thousands of little thumbnails of pictures… you can zoom into any one – and keep zooming… the only hit your machine is taking is the size of your screen – like Google Earth, it’s pulling in information only as it needs it – on the fly.

So, in the SEADRAGON demo they zoomed in through hundreds of tiny, side by side photos lying on the screen – into a tiny glossy car advert… but wait, in the corner of the tiny ad as they zoomed in, were some little car images taken at different angles – zoom in – they’re actually hi-res images with text… pages within pages if you like – almost ad-infinitum if need be… In
another example one photo when zoomed in turned out to be the entire text of a whole book, chapter by chapter size by side – and you just zoomed in smoothly till you got what you wanted… Think about this for your website – instead of lots of pages – you have one page – and just keep zooming in on the bits you’re interested in. A completely different way to think of publications.

At this point the audience of hundreds were starting to smile, but just a little – hardened coders to the last, it would take more than this.

So now take SEADRAGGON and put it together with some or even all of the images available in the world online…. FANTASTIC, want to know all about the Eiffel Tower – thousands of photos taken at different angles and zooms and times of day…… everything you could ever want on your desktop for the zooming – GREAT, but not too easy to navigate!

And none of this is really new – a diffent slant on Microfilm – bigger, better and faster – but then, along comes PHOTOSYNTH … and here I’m coming to the interesting bit…

What the guys at Microsoft have done – is to make software that analyses photos and gives them scoring points for all the features, angles, corners, colours etc and assign a set of numbers to each photo – oh, all totally automatically of course – no human interaction– making it possible to compare one photo with another…. and – wait for it – yes, they can actually tell how one photo relates to another and where the photographer stood when he/she took the shot.
So – take a few pictures of your house at various angles – all the way around and… yes… fire the photos at the software and it is capable of building a “mental image” of the original shape – and you’re right – they’ve managed to put those flat photos in a virtual 3d space- in the order and orientation that they relate to each other.

To explain – imagine you built a plaster model of your house… and on each wall or corner, stick a photo you’ve taken of the real house, hold it there with a pin… and let’s say you’ve a really nice door… as someone moves up to the door in the photo of the front of the house….well, you can’t do this in a real model – but imagine moving close up to the door – and the photo changes from an image of the front of the house, to the close-up of the door…. Imagine an image of the door handle – look toward the handle – image smoothly changes to that image… one could go on ad nauseum.

Don’t get me wrong, we’re not really talking about a guided tour of your house… no – taking a slight leap, we’re talking potentially about a guided tour OF THE WORLD!!!!! (minus boring fields and the sea of course)

That’s today, but what’s REALLY EXCITING is this… some time ago I saw a demo at a university – of a very crude attempt to extract 3d imagery from photos – but this is WAY beyond that… they then briefly showed us a demo… same scenario – a church this time – about 6 photos… move around the photos smoothly and you get an idea of the original church – but that was not good enough – all of a sudden the image smoothly moved AROUND THE OUTSIDE OF THE BULDING IN FULL 3D. Amazingly – they have demo software that figures out the original 3D IMAGE FROM THE PHOTOS and lets you view and pan a complete, solid 3d image!!!! All from a bunch of flat photos and NO human interaction.

Move forward to a different world to the one we have today… let’s say in a few short years at most and most of this is conjecture on my behalf.

Consider something like Microsoft Virtual Earth – consider a system that lets you zoom down to the ground in full 3D, walk around at ground level in full 3D, up alleys, into shops, to anywhere in the world that FLICKR etc have available in images in sufficient number! – Not only THAT but the text that accompanies modern photos (meta information) is used to tap into WIKIPEDIA etc as required and extract useful notes for you.

Imagine sitting at the screen, descending to a shop in a street, going into the shop, let’s say a jewelers shop – and as you zoom along the shelves you spot a ring you like – zoom in and you can see the crystal in every minute detail. Press a button and get complete information about the product – and how long it will take to get to you. Press another – credit card – and it’s yours… within seconds the images changes on the shelf to “sold out”.. and everyone else looking in sees that change.

But won’t all this need HORRENDOUS graphics and power on your PC??? Well, this is 3D –and the gaming industry has long since mastered this stuff.Most modern PCs can run VISTA’s amazing graphics because the gaming industry drove the development of cheap graphics cards.

They’ve had 3D games for a few years and are THAT far off having thought-controlled navigation – headset, sensors, shutter glasses (alternating images to each eye) – and when this all works together seamlessly, which it will at some point in the near future – we’re looking at 3D virtual travel around the world in seconds.

What about adding sound to images – or simply uploading geocoded sound. Before long, most smartphones will have GPS built in – and I’m thinking cameras will too – so photos and any sound will include where they were taken and when!!

But it doesn’t stop THERE…. In 20 years time we’ll not only have photos of just about anywhere relevant – but we’ll ALSO have photos going back 20-30-40 years.. it’s not a big leap to then NOT ONLY move through a virtual 3D landscape – but move through time for heaven’s sake!! Imagine looking at let’s say a city centre – and then smoothly pan back in time to what it looked like, say, 30 years ago… all in a matter of seconds and at the touch of a button.

But you might say… would people REALLY spend their time wandering about in virtual worlds – and that takes me off in another direction. Look up on the web to see how many people are playing World of WarCraft, HALO 3 and any of hundreds of other massive online games…. The answer is YES, people do that. People even discover friends and get married to people they meet in these strangle places.. and the imagery in these worlds is fictional though the people inside are real– what happens when it is all about real places as well?

There are many potential uses for this and you can guarantee that once the fun is over and the novelty wears off – business uses for this will follow en-masse – the point being this goes beyond anything that certainly I’ve seen in science fiction – and its most definitely NOT fiction – it’s mere MONTHS away from slowly beginning to turn into a reality. Of course there are a bundle of issues here not the least being copyright.. but the technology is on the way to make this all happen. The rest, as they say, is up to us.

I hope I’ve somehow managed to convey the awe that those of us who attended this talk felt, not only during the talk, but importantly in the moments after we left as it all started to sink in.