Seeni recently forwarded me the results of this survey, for the "Best companies to Work for in India", conducted by TNS-Mercer-Business Today. Infosys topped the survey as the best company to work for in India. This resulted in some interesting discussion. Some excerpts from the discussion are below. The survey links require subscription to BT. So to provide some background this is an excerpt from the methodology part of the survey: (italics mine)

After much deliberation, BT and knowledge partner Mercer Human Resource Consulting came up with a methodology built around four quadrants (for a note on the methodology itself, see page 78). These relate to HR metrics, employee satisfaction, perceptions of key stakeholders, and HR processes and policies. (each with 15%, 35%, 30%, and 20% weightage respectively)

Then they describe the criteria in each quadrant

We start by looking at the number of employees (which indicates the complexity involved in people management; in fact, to participate in the survey, a company must have at least 200 white-collar employees), attrition rates, average career tenure, promotion rate (that is, total number of promotions as a percentage of the overall headcount), average time for promotion, and go up to things like HR budget and the actual spend as a percentage of total revenues, training budget and spend, and gender ratio. ...

we seek feedback from randomly selected employees on several key issues. These are: leadership, HR processes, transparency, alignment with organisational direction, quality of work life, employee development, and the HR function in the organisation. ....

There are five broad issues that we consider within this quadrant: HR philosophy and policies; recruitment, selection and induction; performance and career management; training and development; rewards and recognition; and culture and work-life balance. ....

...Finally, we ask for the opinions of four important stakeholders in the job market: A company's new recruits, ex-employees, campuses, and search firms.

One final quote about the methodology

Weights were assigned to each quadrant and, as per the methodology, were premised on the need for companies to demonstrate balanced and comprehensive performance across dimensions. The performance of a company on one quadrant was sought to be corroborated by evidence of performance on the other quadrants. The index scores based on quadrant performance and weights were used to propose the ranking order of the companies.

With that background about what the survey is about and how they did it, here are some of my comments :

1. The purpose of any such external survey is not to make the product (engg college, univ, IT company) a better one. (that is just a side effect) The survey is for the end user to make some well informed decisions. If it is not detailed and accurate the purpose is defeated.

2. If the methodology is not good enough to be published in its entirety your survey is not good enough to base my decisions on it. The above survey didn't publish it fully. For example statements like

(The companies were asked to give a large list of employees and the respondents were picked randomly.)

don't do me any good. What is a large list ? All the employees in company ? 10 % of it ? Can the company pick all the employees with the best performance reviews and give it to the survey ?

2. What is the scoring system ? Was it properly explained ? I would say NO. What were the weights for average career tenure or average time for promotion ? We don't know.

3. Forget the ranking table. What is the score of each and every participant in every aspect you chose to evaluate ? Was it published ? How much difference exist between #1 and #2 ? #1 and #131 ? Who leads in criteria A/B/C ? No answers again.

4. Leave the above point. At the least, What is the mean of all the participants, standard deviation among companies ? When you are assigning quantitative numbers as rankings better publish what quantitative numbers you used to arrive at those rankings. Qualitative statements are just not enough. If all of them were at 60% employee satisfaction then who really cares about this whole stuff ?

5. Do people who fund surveys have some interests in the survey results. What is it ? It appears Mercer had some conflict of Interest. So they got some random codes instead of company names. Fair enough. But companies can be easily guessed (at least a few in the top 10) based on their employee strength which was in the details collected. Let us assume those details were not shown to Mercer and the survey is fair.

The article on Infosys mentions that the company helped employees when they were in trouble during the London bombings. There are similar statements about other top 10 companies. I know Infosys gave just week vacation for my cousin to travel to India , get married and return to US, including the travel time, which is bad. Both of these are qualitative statements and don't prove nothing. To be useful enough it has to be quantitative. How many times it helps and how many times does it screw up ?

The employees were selected by the the group that conducted the survey. But what is the sample size. Was the sample size same for infosys and say a startup which had 300 people.

Have you heard of the two monkey problem. Ask two monkeys to type 300 letters. Now there is a subsequence of 50 letters that match in both of them. Ask the same monkeys to type 60000 letters. Now there is a subsequence of 50 letters that match. Which appears more significant ? So, was the sample size good enough to be statistically significant ?

I am not saying they did a mistake in the survey or that the survey was rigged. What I am saying is if they don't publish all the details and all that is available publicly is some ranking table they give me then the results are not verifiable. Hence invalid till proven otherwise. It is not a scientific survey.

It is not even good for the end user to make informed decisions. Can I decide about joining a company ? Can I marry my daughter to a person who works for Infosys ? (Well... may be that one) . If I want to join a company that is highly ranked in work life balance but I don't care much about say training then the survey can't help.

So what good is it for ? If you are in top 10 just enjoy the attention you get. Post it in your website. Project it to your clients. You know, the usual stuff. Otherwise, just forget it. It is not something you should take seriously, atleast in the form it is published now.

PS : I mentioned Infosys not because I don't like the organization. It just happened to be the No, 1.