Posts Tagged ‘google’

More and more content on the internet is 'real time'. Twitter messages, news feeds, pictures, facebook, etc.. Where we used to browse the web for things that have mostly been written in the past, more and more of our internet minutes are spent watching things that 'just happened'.

I see this as a threat to Google, and it wouldn't surprise me if they finally manage to buy Twitter, because Twitter helps them become more 'real time'.

To give an example, I was just trying to update my profile picture on Twitter, and this didn't work. For some reason it refused my pic without an apparent error messsage. Possibly I'm uploading something wrong, so first I googled for 'twitter profile picture' and got this result:

Then, I did the same search on Twitter Search and the result was this:

As you can see, this tells me that in the past 22 minutes, multiple people had this problem. (Ironically you can also see from their avatars that it actually is a problem). The Google results on the other hand, have nothing that is relevant if you take into account the 'now' factor.

This is just an example. Searching for 'current content' is getting more and more relevant. Comments on a live show on tv or an address to the nation by the president; Google is useless in finding these things.

So it is my humble opinion that either a) Google will buy Twitter, Facebook or another 'real time' content site, or b) Google will release an updated Google Search that takes the whole 'now' into account in its search results.

I have been inspired by Cal Evans' mashup experiment, which he did a presentation on at the Dutch PHP Conference last month.

Also, I was annoyed with having to consult many different resources when I plan a business trip.

So I began building frekfly, my own little mashup.

Version 1

The first version, which took me only a few hours to built, offered only a google map, basic weather information, flickr images and currency conversion. To built this, I used the following tools:

• PHP5's Webservices API, which made it possible to consume webservices with only very few lines of code.
• The Flickr API to retrieve images
• Google Maps for the map display
• Some webservicex services, for weather, geographical and currency information.
• Zend Framework, mainly for its Zend_Service_Flickr class, which makes consuming the flickr rest service easy.
• ATK, for its debugging console and file-caching.

It's really easy to construct an application like this in PHP, given that a few lines of code give you access to any available webservice. The first version of the app did not even use a database, everything was retrieved from webservices and cached in files.

Version 2

Then, I wanted more. When I travel for work, I usually need hotels that are located close to the airport, so I looked for a webservice that could provide me with hotel information. I requested access to the API at expedia, but they have a manual approval process and I'm still waiting for a response. Luckily the people at booking.com were a lot more helpful and they provided me with an iframe based interface to their hotel reservation engine, and they even styled their pages according to the design (well ok, that's not really a design yet, is it ) of frekfly.

The only thing missing from their API was the ability to search by latitude and longitude, which was my close-to-the-airport criterium. However, they kindly send me a dump of their hotel database including the coordinates.

So in version 2 I had to add a database, and I used the database abstraction layer of ATK to easily access it.

Version 2 is what is currently online. I may post some code examples of the webservice interfaces later on, because I found this to be a very instructive experience.

Future plans

In the next version, I want to add other nifty web-two-point-oh features to the site, including the ability to have visitors enter comments and details, so they can help complete the airport information. (such as 'are there better restaurants before or after security?'). Here I plan to use the JSON functionality of Zend Framework, to make this very 2.0-ish.

Issues to deal with

One of the major disadvantages of mashups is the fact that you're dependent on external systems. At night, the site is hardly usable because webservicex is very busy at that time and gives a lot of timeouts. Of course, this is countered by caching, but I can't cache every output for every airport on the planet, so caching is based on a 'retrieve the first time its needed' approach. So that's a disadvantage at this point.

Another interesting issue you have to deal with when writing mashups is standardization in naming, or rather, the lack thereof. For example, I consult 2 different webservices and a few file based airport resources to get airport information, but each of them use different names for the same airport. Where possible, I use the official 3-letter IATA code, but some of them do not support that.

So instead of actually matching data from different sources, you have to use a more probabilistic approach, where you say "hmm, 'Schiphol Airport Amsterdam' and 'Amsterdam, Schiphol' are probably the same airport.

I had a similar issue with the weather service; it doesn't support latitude and longitude, so what it does is 'guess' what the nearest weatherstation is, based on the airportname, the nearest town name, the biggest city close to the airport and finally the country capital if all else fails.

These are all issues to deal with in a mashup-based application, and I would encourage everyone to built a mashup at some point, because it teaches you things that you don't usually encounter with standard mysql+php based websites.

Recently, Google opened a search engine that will appeal to all developers: Google Code Search, for searching through all available source code on the web.

While this is great to find example usages of functions, or to find security vulnerabilities, a far more fun thing to do is delve into the frustrations developers encounter while developing code. Frustrations that are filtered out by the compiler, and are hidden away into source code comments.

This search for example is a nice one, and I had a good laugh at the first result of this search.

You can also see how programmers think about certain big corporations when using their API's.

It's also fun to see how much faith developers have in their code.

Finally, it's nice to have a way to see how much legends such as 'foo' are actually used, and that some developers even put prayers in their programs.

One tip: if you search for PHP variables, prepend the $ with a , as Google Code Search supports regular expressions, so you have to escape certain characters.

When in october 2004 Peter and I took on the idea of developing epointment.com, we thought we had all the time in the world, given the fact that at that time most online calendering applications were rather rubish.

Over the past few years, several new sites have been created for this same purpose, and when a few months back the rumours about Google Calendar started, I was initially horrified. I felt what I think employees of Netscape must have felt when Microsoft released Internet Explorer.

Funny thing is, it's hardly ever about money. Netscape was already freely downloadable when Internet Explorer was released. Similarly epointment is as much free as Google Calendar is (our investments are to be won back by providing professional services). This is more about how given enough weight, any market can be taken. Internet Explorer didn't crush Netscape because it was a better browser. Microsoft just was a bigger player with better cards.

Now we don't claim to be better than Google Calendar, that would be quite presumptuous, given that probably they employ some of the smartest minds in the industry and have a lot bigger team than the two of us working on epointment. But my point is that neither price, nor quality, matter, when the elephants walk by. The fact that it is 'Google' Calendar and not 'Just Any Calendar', makes it Good. (I sometimes have the same feeling about art. Some art is art only because of the signature on it). And don't get me wrong, Google Calendar actually *is* good. And competition is a good thing anyway, it gives people choice.

And, instead of sobbing, which never really helps anyway , mice like us should do the only sensible thing: distinguish ourselves. We have many ideas that we will roll out in the foreseeable future, but we already do it very subtly. While Ajax is hype and everybody makes sure Ajax is part of their product description, we stay away from the term. We actually use the technology (using prototype and script.aculo.us if you're interested), but you will have a hard time finding it.

Another choice we made is to make the agenda not look like the digital version of the schedule of a trained secretary, but make it something that is more like the type of calendar that hangs on your mother's wall. Keep it simple, so to say. Most people don't really need a digital secretary, they just need a way of keeping track of (and remembering) their appointments.

So no, you cannot sync your appointments in epointment yet, we'll focus on ease of use first (and how much do the non-techies sync their agenda's in real life anyway? )

One thing we did realize in the past months do is that looks do matter. The initial layout (well, let's rather call it a complete lack of layout) was, in hindsight, terrible. So yesterday we finally released a new version of the site that we feel looks a bit better. Now all we have to do is stay away from the elephant's feet and run right under them and finish first.

Oh and, because we are geeks after all, we just had to implement easter eggs. Not that anyone would ever find them, but writing them was fun in itself.

After I noticed the internationalisation quirks in google, I found several other things that seem to suggest that google starts to 'think' for the user.

If on google.com, I perform a regular search for 'pagani zonda', google first presents me three image results, followed by the regular results. If I search directly on google images, I first find images that are of far less quality than the three I got on my normal search result page. Seems almost as if they were hand-picked.

When I search for 'Jennifer Aniston' however, it first presents a set of news links, then the regular results. No pics this time.

Both Jennifer Aniston and Pagani Zonda have news items and images related to them. But somehow it decides that I would want to have pictures of a Pagani, but news about Jennifer.

(and I'm not using the personalization feature).

Another thing I noticed, when I search for the word 'gem', the first 10 results are 'compromised'. 3 out of the 10 results are results for the word 'jem'; As if Google not only wants to suggests that the word is spelled wrong, but it is so sure about that, that it makes 33% of the first results page reflect this alternative spelling (even though 'gem' is a perfectly legal English word).

This is different behaviour than when I search for many other words, when it just uses the regular 'did you mean ...' suggestion.

Searching for 'jem' first presents a google music search result, before displaying 'regular' results.

Luckily, searching for 'php' still seems to be untouched.

I find this scary. Where google used to be a page rank based objective search engine, presenting just what it found in its indexes, it's now starting to become an engine that influences what I find, based on assumptions it makes, probably from all the search stats they are collecting.

Horror story: once the entire world uses google, they control our very access to information, and by manipulating the results based on our own 'preferences', they essentially control what information we do or do not consume. I guess this is material for a new Dan Brown novel.

I'm facing a challenge with a customer running a site we built on a hosting environment that is quite restrictive. We used to use htdig to add search functionality, but the hoster doesn't allow execution of binaries.

I've taken a look at several search engine implementations, but so far, none match all requirements:

• Zend Framework's lucene search: requires PHP5, but we have to run it on PHP4

• phpdig: uses exec() to perform the search, but we are running in safe mode
• google: layout not flexible enough
• htdig: requires execution of binaries, which is not allowed
• perlfect: requires execution of perl, which is not allowed

Maybe isearch is an option but I cannot find a lot of info about it.

The requirements are simple, but restrictive:

• Should be usable/integratable in a PHP4 based site

• Should run with safe mode on
• Layout should be customisable
• Should not require execution of perl or binaries, at least not for the search part (uploading an index that is created on a separate machine is somewhat acceptable although not prefered)
• Should be spider based, so no database query based search engine

Free Software/Open Source/Free as in Free Beer would be nice, but is no strict requirement.

Anybody any suggestion? What are you guys using?