Yandex.com – the new search engine from Russia

Yandex.com is the recently launched English version of the successful Russian search engine, and it is good. Here’s a bit more about it:

The history of Yandex

The Yandex search engine was originally founded in 1993 (so it pre-dates Google). The full name initially stood for “Yet Another Index”, and the Russian for “I” is “Ya”. So “Yandex”=”Index” – geddit?

Because it was Russian based, it was yandex.ru (still the engine for the Russian language) and quickly dominated search for all Russian language queries.

However in 2006, Google entered the Russian search market and started to gain market share going from 5% to 30% in a couple of years.

Part of the problem was down to Yandex counting all links, paid and natural. They do this on the grounds that very few links on the web are natural, and that Google’s distinction between directly paid links (bad), and paying people to write lots of artificial articles linking to you (sorta OK), was a nonsense. However, the paid links did distort their index a bit. But Yandex fought back: asinternational SEO expert Andy Atkins-Krüger explains, Yandex started venturing into machine learning to improve their engine.

In 2009 they launched their MatrixNet algo, which is based on machine learning, and their search quality leapt forward, enabling them to claw back market share from Google in Russia, as they were serving better results than Google.

MatrixNet and Machine Learning

I recommend reading the following page which explains how MatrixNet works.  They use human assessors to evaluate a sample of web pages as good or bad. This learning sample is fed into their engine to analyse these pages. They explain the process picturesquely in the following way:

Think of teaching a machine how to pick the most delicious apples. First, assessors take a bite of each apple in a ‘tasting crate’ and put all tasty apples to the right and all sour apples to the left. This crate contains all sorts of apples in the same proportion as they are likely to grow in the garden. A machine cannot taste apples, but it can analyze their properties, like size, color, sugar content, firmness, presence or absence of a leaf. The tasting crate is a learning sample, which allows the machine to learn to select the apples with the winning combination of properties: size, color, sweetness and firmness. Errors are unavoidable, though. For instance, if a machine does not have any information about insect larvae, the best apples it has selected might hide a worm. To minimize the probability of error, a machine needs to consider a maximum number of apples’ properties.

They then go onto explain some of the problems, such as “overfitting”  and how they correct them.

If the above process sounds familiar, it’s because Google has decided to copy them, launching the Google Panda update in Feb 2011, which is based on machine learning and has been rolled out globally.

Yandex however remains far more experienced at machine learning than Google, they’ve been doing it for a while and have learnt all the pitfalls (a process where Google is still in the nursery). They launched their English language search engine Yandex.com in April 2010. It was probably the move by Yandex into the English market that prompted Google to start work on redoing their algorithm, especially as complaints were being aired about spam in Google’s results (due to the pesky problem of people paying to manipulate the index).

The AGS-30 filter

The AGS-30 is a Russian automatic grenade launcher – which should give you an idea what this filter is about! It penalises sites that it thinks are poor quality, and is part of the MatrixNet algo. I’ve seen this in action for real on Yandex.com – when searching for “Warrior Forum”, I simply could not find it! The problem was that the Warrior’s home page was going through too many redirects, and while G still listed it despite the problems, Yandex nuked it! Please see the following page for more on how SEO for Yandex works.

In my opinion the Yandex.com engine provides better results than Google, andfar better results than Bing. Try it out and see. Plus there is the added benefit of no ads cluttering it up.

I’ve set it up as my default search engine in the Chrome browser (in my opinion Chrome is the best web browser out there – Google still does some things excellently!). Here’s how to do it:

Click on the tool icon on the top right corner of the Chrome Browser and select “Options”. The Settings page will load in your browser. Next click “Manage Search Engines” and simply select Yandex as your default. And you are done!

I predict you will find searching with Yandex a pleasanter experience than searching with Google. I’ve yet to have a bad set of results served up. I don’t think it will be too long before their English language market share starts increasing.